Train the Neural Network
- This is where all the previous parts of the Crash Course come together.
- Follow along from previous post
- Train the previously defined network with data
from mxnet import nd, gluon, init, autograd
from mxnet.gluon import nn
from mxnet.gluon.data.vision import datasets, transforms
from IPython import display
import matplotlib.pyplot as plt
import time
import mxnet as mx
from mxnet import profiler
profiler.set_config(profile_all=True, aggregate_stats=True, filename='profile_output.json')
Get Data
- Handwritten digit MNIST dataset is commonly used datasets in deep learning. But it is rather simple.
- In this case, we use more complicated dataset called FashionMNIST.
- Goal is to classify clothing type.
- Dataset can be automatically downloaded through Gluon’s data.vision.datasets module.
mnist_train = datasets.FashionMNIST(train=True)
X, y = mnist_train[0]
('X shape: ', X.shape, 'X dtype', X.dtype, 'y:', y)
('X shape: ', (28, 28, 1), 'X dtype', numpy.uint8, 'y:', 2)
- Each example in the dataset is a 28x28 grey image presented as NDArray with the shape of format (height, width, channel)
text_labels = ['t-shirt', 'trouser', 'pullover', 'dress', 'coat', 'sandal', 'shirt', 'sneaker', 'bag', 'ankle boot']
X, y = mnist_train[0:10]
# plot images
display.set_matplotlib_formats('png')
_, figs = plt.subplots(1, X.shape[0], figsize=(15, 15))
for f, x, yi in zip (figs, X, y):
# 3D -> 2D by removing the last channel dim
f.imshow(x.reshape((28, 28)).asnumpy())
ax = f.axes
ax.set_title(text_labels[int(yi)])
ax.title.set_fontsize(14)
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
plt.show()
- To feed data into Gluon model, we need to convert the images to (channel, height, width) format. This can be done with transforms.ToTensor().
- We also need to normalize all pixel values with a mean value of 0.13 and std_dev of 0.31. This can be done with transforms.Normalize()
- We can chain these transforms together and apply it to the first element of the data pair.
transformer = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(0.13, 0.31)]
)
mnist_train = mnist_train.transform_first(transformer)
- FashionMNIST is a subclass of gluon.data.Dataset which defines how to get the i-th example.
- Here we use gluon.data.DataLoader and spin off 4 worker threads to process data in paralle.
batch_size = 256
train_data = gluon.data.DataLoader(
mnist_train, batch_size=batch_size, shuffle=True, num_workers=4)
- The returned train_data is an iteratable object that yields batches of images and label pairs.
for data, label in train_data:
print(data.shape, label.shape)
break
(256, 1, 28, 28) (256,)
- Finally, we create a validation dataset and data loader
mnist_valid = gluon.data.vision.FashionMNIST(train=False)
valid_data = gluon.data.DataLoader(
mnist_valid.transform_first(transformer),
batch_size=batch_size, num_workers=4)
Define the Model
- We reimplement the same LeNet model introduced before.
- The difference here is that the weight initialization method is changed to Xavier which is a popular choice for deep convolutional neural network
net = nn.Sequential()
net.add(
nn.Conv2D(channels=6, kernel_size=5, activation='relu'),
nn.MaxPool2D(pool_size=2, strides=2),
nn.Conv2D(channels=16, kernel_size=3, activation='relu'),
nn.MaxPool2D(pool_size=2, strides=2),
nn.Flatten(),
nn.Dense(120, activation="relu"),
nn.Dense(84, activation="relu"),
nn.Dense(10)
)
net.initialize(init=init.Xavier())
- That was the network and it’s layers, now we need to define the loss function and optimization method for training
- We will use standard softmax cross entropy loss for classification problems.
- It first performs softmax on the output to obtain the prdeicted probability, and then compares the label with the cross entropy (one-hot encoding I think)
- Also, note that this is the same done in the Udacity class.
softmax_cross_entropy = gluon.loss.SoftmaxCrossEntropyLoss()
- Optimization Method is the standard stochastic gradient descent with constant learning rate of 0.1
# This code is failing in mxnet 1.2.1
trainer = gluon.Trainer(net.collect_params(), 'sgd', {'learning_rate' : 0.1, 'lazy_update': False})#, lazy_update=True)
- The trainer is created with all params (weights and gradients) in net.
- From now on, we need to only call step() method to update its weights.
Train
- We create an auxiliary function to calculate the model accuracy
def acc(output, label):
# output: (batch, num_output) float32 ndarray
# label: (batch, ) int32 ndarray
return (output.argmax(axis=1) ==
label.astype('float32')).mean().asscalar()
- Implement the complete training loop
#mx.profiler.set_config(profile_all=True, aggregate_stats=True, filename='profile_output_test.json')
#mx.profiler.set_config(filename='profile_output_test.json')
count = 0
mx.profiler.set_state('run')
for epoch in range(2):
train_loss, train_acc, valid_acc = 0., 0., 0.
tic = time.time()
for data, label in train_data:
# forward + backward
with autograd.record():
output = net(data)
loss = softmax_cross_entropy(output, label)
loss.backward()
# update parameters
trainer.step(batch_size)#, lazy_update=True)
# calculate training metrics
train_loss += loss.mean().asscalar()
train_acc += acc(output, label)
mx.nd.waitall()
print(mx.profiler.dumps())
#mx.profiler.pause()
count += 1
# calc validation accuracy
for data, label in valid_data:
valid_acc += acc(net(data), label)
print("Epoch %d: loss %.3f, train_acc %.3f, test acc %.3f, in %.1f sec" % (
epoch, train_loss/len(train_data), train_acc/len(train_data),
valid_acc/len(valid_data), time.time() - tic
)
)
mx.profiler.set_state('stop')
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37254 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3306 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 470 1871.0150 0.3060 73.5850 3.9809
MXNDArrayCreateEx 551 18.8610 0.0310 0.0640 0.0342
MXNDArrayWaitAll 471 10161.7070 5.6130 24.8060 21.5747
MXNDArraySyncCopyToCPU 1022 8066.7949 0.1180 24.1980 7.8931
MXNDArrayGetDType 1022 0.3560 0.0000 0.0010 0.0003
MXNDArrayGetShape 2595 1.2030 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1102 9.6770 0.0070 0.0280 0.0088
MXNDArrayFree 12273 125.5670 0.0000 0.9140 0.0102
MXNet C API Calls 54797 542.1540 487.3580 542.1540 27.3980
MXNet C API Concurrency 109593 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 942 0.4510 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4710 2.4350 0.0000 0.0230 0.0005
MXNDArraySetGradState 4710 2.4020 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 942 0.4060 0.0000 0.0010 0.0004
MXAutogradBackwardEx 471 256.0930 0.5250 0.6810 0.5437
MXNDArrayGetContext 7634 3.2950 0.0000 0.0070 0.0004
MXImperativeInvokeEx 15881 403.5790 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1884 4568.6821 0.7010 3.7330 2.4250
FullyConnected 3306 310.2470 0.0080 0.3430 0.0938
argmax 1102 21.4350 0.0050 0.0480 0.0195
_backward_pick 942 14.7070 0.0130 0.0260 0.0156
_backward_mul_scalar 942 4.0580 0.0030 0.0160 0.0043
DeleteVariable 40560 288.3490 0.0010 0.9010 0.0071
Flatten 1102 45.5040 0.0080 0.1120 0.0413
Activation 4408 198.3770 0.0030 0.3430 0.0450
SetValueOp 942 6.0010 0.0050 0.0180 0.0064
pick 942 7.3280 0.0060 0.0280 0.0078
Pooling 2204 3013.7991 0.0640 3.1720 1.3674
Convolution 2204 12509.4717 0.2260 13.2050 5.6758
CopyCPU2CPU 1102 6.3130 0.0030 0.0340 0.0057
log_softmax 942 28.8560 0.0150 0.0500 0.0306
_backward_copy 942 48.3820 0.0220 0.0760 0.0514
_backward_Convolution 1884 17332.5391 2.6370 12.4340 9.1999
_mul_scalar 942 3.3120 0.0020 0.0230 0.0035
_backward_log_softmax 942 15.6330 0.0090 0.0400 0.0166
mean 2986 56.8470 0.0080 0.0530 0.0190
_backward_mean 942 10.2220 0.0080 0.1030 0.0109
_backward_FullyConnected 2826 522.7040 0.0280 0.5940 0.1850
WaitForVar 2044 18.0920 0.0060 0.0280 0.0089
broadcast_equal 1102 5.4350 0.0030 0.0100 0.0049
sgd_update 9420 90.5080 0.0020 0.0640 0.0096
_backward_Activation 3768 274.3250 0.0070 0.3900 0.0728
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37328 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3312 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 471 1875.0341 0.3060 73.5850 3.9810
MXNDArrayCreateEx 552 18.8960 0.0310 0.0640 0.0342
MXNDArrayWaitAll 472 10186.3789 5.6130 24.8060 21.5813
MXNDArraySyncCopyToCPU 1024 8084.5708 0.1180 24.1980 7.8951
MXNDArrayGetDType 1024 0.3570 0.0000 0.0010 0.0003
MXNDArrayGetShape 2600 1.2040 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1104 9.6960 0.0070 0.0280 0.0088
MXNDArrayFree 12296 125.8260 0.0000 0.9140 0.0102
MXNet C API Calls 54904 542.2610 487.3580 542.2610 27.4515
MXNet C API Concurrency 109807 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 944 0.4520 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4720 2.4640 0.0000 0.0230 0.0005
MXNDArraySetGradState 4720 2.4080 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 944 0.4080 0.0000 0.0010 0.0004
MXAutogradBackwardEx 472 256.6920 0.5250 0.6810 0.5438
MXNDArrayGetContext 7648 3.3010 0.0000 0.0070 0.0004
MXImperativeInvokeEx 15912 404.4320 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1888 4580.9038 0.7010 3.7500 2.4263
FullyConnected 3312 311.1820 0.0080 0.3430 0.0940
argmax 1104 21.4860 0.0050 0.0480 0.0195
_backward_pick 944 14.7450 0.0130 0.0260 0.0156
_backward_mul_scalar 944 4.0670 0.0030 0.0160 0.0043
DeleteVariable 40640 288.9830 0.0010 0.9010 0.0071
Flatten 1104 45.6010 0.0080 0.1120 0.0413
Activation 4416 198.9590 0.0030 0.3430 0.0451
SetValueOp 944 6.0180 0.0050 0.0180 0.0064
pick 944 7.3470 0.0060 0.0280 0.0078
Pooling 2208 3023.3240 0.0640 3.1720 1.3693
Convolution 2208 12531.7891 0.2260 13.2050 5.6756
CopyCPU2CPU 1104 6.3250 0.0030 0.0340 0.0057
log_softmax 944 28.9380 0.0150 0.0500 0.0307
_backward_copy 944 48.4900 0.0220 0.0760 0.0514
_backward_Convolution 1888 17373.5645 2.6370 12.4520 9.2021
_mul_scalar 944 3.3220 0.0020 0.0230 0.0035
_backward_log_softmax 944 15.6810 0.0090 0.0400 0.0166
mean 2992 56.9790 0.0080 0.0530 0.0190
_backward_mean 944 10.2490 0.0080 0.1030 0.0109
_backward_FullyConnected 2832 524.4380 0.0280 0.5940 0.1852
WaitForVar 2048 18.1350 0.0060 0.0280 0.0089
broadcast_equal 1104 5.4480 0.0030 0.0100 0.0049
sgd_update 9440 90.8100 0.0020 0.0640 0.0096
_backward_Activation 3776 275.1580 0.0070 0.3900 0.0729
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37402 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3318 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 472 1878.9390 0.3060 73.5850 3.9808
MXNDArrayCreateEx 553 18.9300 0.0310 0.0640 0.0342
MXNDArrayWaitAll 473 10210.9893 5.6130 24.8060 21.5877
MXNDArraySyncCopyToCPU 1026 8102.4761 0.1180 24.1980 7.8972
MXNDArrayGetDType 1026 0.3580 0.0000 0.0010 0.0003
MXNDArrayGetShape 2605 1.2070 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1106 9.7130 0.0070 0.0280 0.0088
MXNDArrayFree 12319 126.0700 0.0000 0.9140 0.0102
MXNet C API Calls 55011 542.3680 487.3580 542.3680 27.5050
MXNet C API Concurrency 110021 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 946 0.4530 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4730 2.4690 0.0000 0.0230 0.0005
MXNDArraySetGradState 4730 2.4160 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 946 0.4100 0.0000 0.0010 0.0004
MXAutogradBackwardEx 473 257.2280 0.5250 0.6810 0.5438
MXNDArrayGetContext 7662 3.3070 0.0000 0.0070 0.0004
MXImperativeInvokeEx 15943 405.2070 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1892 4593.1709 0.7010 3.7500 2.4277
FullyConnected 3318 312.0800 0.0080 0.3430 0.0941
argmax 1106 21.5360 0.0050 0.0480 0.0195
_backward_pick 946 14.7810 0.0130 0.0260 0.0156
_backward_mul_scalar 946 4.0750 0.0030 0.0160 0.0043
DeleteVariable 40720 289.5980 0.0010 0.9010 0.0071
Flatten 1106 45.6820 0.0080 0.1120 0.0413
Activation 4424 199.4900 0.0030 0.3430 0.0451
SetValueOp 946 6.0340 0.0050 0.0180 0.0064
pick 946 7.3660 0.0060 0.0280 0.0078
Pooling 2212 3032.7900 0.0640 3.1720 1.3711
Convolution 2212 12554.0596 0.2260 13.2050 5.6754
CopyCPU2CPU 1106 6.3360 0.0030 0.0340 0.0057
log_softmax 946 29.0150 0.0150 0.0500 0.0307
_backward_copy 946 48.5880 0.0220 0.0760 0.0514
_backward_Convolution 1892 17414.5625 2.6370 12.4700 9.2043
_mul_scalar 946 3.3300 0.0020 0.0230 0.0035
_backward_log_softmax 946 15.7280 0.0090 0.0400 0.0166
mean 2998 57.1040 0.0080 0.0530 0.0190
_backward_mean 946 10.2750 0.0080 0.1030 0.0109
_backward_FullyConnected 2838 526.1290 0.0280 0.5940 0.1854
WaitForVar 2052 18.1760 0.0060 0.0280 0.0089
broadcast_equal 1106 5.4600 0.0030 0.0100 0.0049
sgd_update 9460 91.0470 0.0020 0.0640 0.0096
_backward_Activation 3784 276.0180 0.0070 0.3900 0.0729
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37476 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3324 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 473 1882.8390 0.3060 73.5850 3.9806
MXNDArrayCreateEx 554 18.9640 0.0310 0.0640 0.0342
MXNDArrayWaitAll 474 10235.7041 5.6130 24.8060 21.5943
MXNDArraySyncCopyToCPU 1028 8120.3311 0.1180 24.1980 7.8992
MXNDArrayGetDType 1028 0.3580 0.0000 0.0010 0.0003
MXNDArrayGetShape 2610 1.2100 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1108 9.7290 0.0070 0.0280 0.0088
MXNDArrayFree 12342 126.2940 0.0000 0.9140 0.0102
MXNet C API Calls 55118 542.4750 487.3580 542.4750 27.5585
MXNet C API Concurrency 110235 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 948 0.4540 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4740 2.4750 0.0000 0.0230 0.0005
MXNDArraySetGradState 4740 2.4200 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 948 0.4110 0.0000 0.0010 0.0004
MXAutogradBackwardEx 474 257.7610 0.5250 0.6810 0.5438
MXNDArrayGetContext 7676 3.3120 0.0000 0.0070 0.0004
MXImperativeInvokeEx 15974 405.9780 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1896 4605.3428 0.7010 3.7500 2.4290
FullyConnected 3324 312.9420 0.0080 0.3430 0.0941
argmax 1108 21.5850 0.0050 0.0480 0.0195
_backward_pick 948 14.8160 0.0130 0.0260 0.0156
_backward_mul_scalar 948 4.0840 0.0030 0.0160 0.0043
DeleteVariable 40800 290.1710 0.0010 0.9010 0.0071
Flatten 1108 45.7640 0.0080 0.1120 0.0413
Activation 4432 200.0210 0.0030 0.3430 0.0451
SetValueOp 948 6.0500 0.0050 0.0180 0.0064
pick 948 7.3840 0.0060 0.0280 0.0078
Pooling 2216 3042.2451 0.0640 3.1720 1.3729
Convolution 2216 12576.2920 0.2260 13.2050 5.6752
CopyCPU2CPU 1108 6.3470 0.0030 0.0340 0.0057
log_softmax 948 29.0910 0.0150 0.0500 0.0307
_backward_copy 948 48.6840 0.0220 0.0760 0.0514
_backward_Convolution 1896 17455.7207 2.6370 12.5610 9.2066
_mul_scalar 948 3.3390 0.0020 0.0230 0.0035
_backward_log_softmax 948 15.7740 0.0090 0.0400 0.0166
mean 3004 57.2470 0.0080 0.0530 0.0191
_backward_mean 948 10.3010 0.0080 0.1030 0.0109
_backward_FullyConnected 2844 527.8120 0.0280 0.5940 0.1856
WaitForVar 2056 18.2170 0.0060 0.0280 0.0089
broadcast_equal 1108 5.4720 0.0030 0.0100 0.0049
sgd_update 9480 91.3450 0.0020 0.0640 0.0096
_backward_Activation 3792 276.8340 0.0070 0.3900 0.0730
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37550 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3330 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 474 1886.7330 0.3060 73.5850 3.9804
MXNDArrayCreateEx 555 18.9970 0.0310 0.0640 0.0342
MXNDArrayWaitAll 475 10260.2012 5.6130 24.8060 21.6004
MXNDArraySyncCopyToCPU 1030 8138.1699 0.1180 24.1980 7.9011
MXNDArrayGetDType 1030 0.3590 0.0000 0.0010 0.0003
MXNDArrayGetShape 2615 1.2130 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1110 9.7460 0.0070 0.0280 0.0088
MXNDArrayFree 12365 126.5300 0.0000 0.9140 0.0102
MXNet C API Calls 55225 542.5820 487.3580 542.5820 27.6120
MXNet C API Concurrency 110449 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 950 0.4560 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4750 2.4800 0.0000 0.0230 0.0005
MXNDArraySetGradState 4750 2.4240 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 950 0.4120 0.0000 0.0010 0.0004
MXAutogradBackwardEx 475 258.2930 0.5250 0.6810 0.5438
MXNDArrayGetContext 7690 3.3180 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16005 406.7510 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1900 4617.3179 0.7010 3.7500 2.4302
FullyConnected 3330 313.8490 0.0080 0.3430 0.0942
argmax 1110 21.6350 0.0050 0.0480 0.0195
_backward_pick 950 14.8510 0.0130 0.0260 0.0156
_backward_mul_scalar 950 4.0920 0.0030 0.0160 0.0043
DeleteVariable 40880 290.7580 0.0010 0.9010 0.0071
Flatten 1110 45.8430 0.0080 0.1120 0.0413
Activation 4440 200.5530 0.0030 0.3430 0.0452
SetValueOp 950 6.0630 0.0050 0.0180 0.0064
pick 950 7.4030 0.0060 0.0280 0.0078
Pooling 2220 3051.7500 0.0640 3.1720 1.3747
Convolution 2220 12598.4551 0.2260 13.2050 5.6750
CopyCPU2CPU 1110 6.3590 0.0030 0.0340 0.0057
log_softmax 950 29.1680 0.0150 0.0500 0.0307
_backward_copy 950 48.7830 0.0220 0.0760 0.0514
_backward_Convolution 1900 17496.6133 2.6370 12.5610 9.2087
_mul_scalar 950 3.3470 0.0020 0.0230 0.0035
_backward_log_softmax 950 15.8230 0.0090 0.0400 0.0167
mean 3010 57.3710 0.0080 0.0530 0.0191
_backward_mean 950 10.3260 0.0080 0.1030 0.0109
_backward_FullyConnected 2850 529.5070 0.0280 0.5940 0.1858
WaitForVar 2060 18.2580 0.0060 0.0280 0.0089
broadcast_equal 1110 5.4850 0.0030 0.0100 0.0049
sgd_update 9500 91.5800 0.0020 0.0640 0.0096
_backward_Activation 3800 277.6920 0.0070 0.3900 0.0731
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37624 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3336 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 475 1890.6300 0.3060 73.5850 3.9803
MXNDArrayCreateEx 556 19.0310 0.0310 0.0640 0.0342
MXNDArrayWaitAll 476 10284.6680 5.6130 24.8060 21.6064
MXNDArraySyncCopyToCPU 1032 8155.9102 0.1180 24.1980 7.9030
MXNDArrayGetDType 1032 0.3600 0.0000 0.0010 0.0003
MXNDArrayGetShape 2620 1.2160 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1112 9.7620 0.0070 0.0280 0.0088
MXNDArrayFree 12388 126.7740 0.0000 0.9140 0.0102
MXNet C API Calls 55332 542.6890 487.3580 542.6890 27.6655
MXNet C API Concurrency 110663 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 952 0.4570 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4760 2.4850 0.0000 0.0230 0.0005
MXNDArraySetGradState 4760 2.4280 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 952 0.4140 0.0000 0.0010 0.0004
MXAutogradBackwardEx 476 258.8260 0.5250 0.6810 0.5438
MXNDArrayGetContext 7704 3.3250 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16036 407.5240 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1904 4629.4180 0.7010 3.7500 2.4314
FullyConnected 3336 314.7580 0.0080 0.3430 0.0944
argmax 1112 21.6840 0.0050 0.0480 0.0195
_backward_pick 952 14.8870 0.0130 0.0260 0.0156
_backward_mul_scalar 952 4.1010 0.0030 0.0160 0.0043
DeleteVariable 40960 291.3630 0.0010 0.9010 0.0071
Flatten 1112 45.9230 0.0080 0.1120 0.0413
Activation 4448 201.0880 0.0030 0.3430 0.0452
SetValueOp 952 6.0770 0.0050 0.0180 0.0064
pick 952 7.4220 0.0060 0.0280 0.0078
Pooling 2224 3061.1689 0.0640 3.1720 1.3764
Convolution 2224 12620.4971 0.2260 13.2050 5.6747
CopyCPU2CPU 1112 6.3700 0.0030 0.0340 0.0057
log_softmax 952 29.2460 0.0150 0.0500 0.0307
_backward_copy 952 48.8810 0.0220 0.0760 0.0513
_backward_Convolution 1904 17537.3711 2.6370 12.5610 9.2108
_mul_scalar 952 3.3570 0.0020 0.0230 0.0035
_backward_log_softmax 952 15.8690 0.0090 0.0400 0.0167
mean 3016 57.4970 0.0080 0.0530 0.0191
_backward_mean 952 10.3530 0.0080 0.1030 0.0109
_backward_FullyConnected 2856 531.2060 0.0280 0.5940 0.1860
WaitForVar 2064 18.2990 0.0060 0.0280 0.0089
broadcast_equal 1112 5.4970 0.0030 0.0100 0.0049
sgd_update 9520 91.8800 0.0020 0.0640 0.0097
_backward_Activation 3808 278.5000 0.0070 0.3900 0.0731
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37698 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3342 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 476 1894.5330 0.3060 73.5850 3.9801
MXNDArrayCreateEx 557 19.0640 0.0310 0.0640 0.0342
MXNDArrayWaitAll 477 10306.4551 5.6130 24.8060 21.6068
MXNDArraySyncCopyToCPU 1034 8173.7222 0.1180 24.1980 7.9050
MXNDArrayGetDType 1034 0.3610 0.0000 0.0010 0.0003
MXNDArrayGetShape 2625 1.2190 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1114 9.7780 0.0070 0.0280 0.0088
MXNDArrayFree 12411 127.0040 0.0000 0.9140 0.0102
MXNet C API Calls 55439 542.7960 487.3580 542.7960 27.7190
MXNet C API Concurrency 110877 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 954 0.4590 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4770 2.4860 0.0000 0.0230 0.0005
MXNDArraySetGradState 4770 2.4310 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 954 0.4150 0.0000 0.0010 0.0004
MXAutogradBackwardEx 477 259.3590 0.5250 0.6810 0.5437
MXNDArrayGetContext 7718 3.3290 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16067 408.3020 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1908 4640.1870 0.7010 3.7500 2.4320
FullyConnected 3342 315.7010 0.0080 0.3430 0.0945
argmax 1114 21.7330 0.0050 0.0480 0.0195
_backward_pick 954 14.9230 0.0130 0.0260 0.0156
_backward_mul_scalar 954 4.1100 0.0030 0.0160 0.0043
DeleteVariable 41040 291.9310 0.0010 0.9010 0.0071
Flatten 1114 46.0020 0.0080 0.1120 0.0413
Activation 4456 201.6200 0.0030 0.3430 0.0452
SetValueOp 954 6.0920 0.0050 0.0180 0.0064
pick 954 7.4400 0.0060 0.0280 0.0078
Pooling 2228 3070.5801 0.0640 3.1720 1.3782
Convolution 2228 12642.6738 0.2260 13.2050 5.6744
CopyCPU2CPU 1114 6.3810 0.0030 0.0340 0.0057
log_softmax 954 29.3250 0.0150 0.0500 0.0307
_backward_copy 954 48.9790 0.0220 0.0760 0.0513
_backward_Convolution 1908 17574.3574 2.6370 12.5610 9.2109
_mul_scalar 954 3.3660 0.0020 0.0230 0.0035
_backward_log_softmax 954 15.9150 0.0090 0.0400 0.0167
mean 3022 57.7010 0.0080 0.0730 0.0191
_backward_mean 954 10.3790 0.0080 0.1030 0.0109
_backward_FullyConnected 2862 532.8900 0.0280 0.5940 0.1862
WaitForVar 2068 18.3410 0.0060 0.0280 0.0089
broadcast_equal 1114 5.5080 0.0030 0.0100 0.0049
sgd_update 9540 92.1640 0.0020 0.0640 0.0097
_backward_Activation 3816 279.0540 0.0070 0.3900 0.0731
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37772 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3348 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 477 1898.4270 0.3060 73.5850 3.9799
MXNDArrayCreateEx 558 19.0980 0.0310 0.0640 0.0342
MXNDArrayWaitAll 478 10328.3584 5.6130 24.8060 21.6074
MXNDArraySyncCopyToCPU 1036 8188.5991 0.1180 24.1980 7.9041
MXNDArrayGetDType 1036 0.3620 0.0000 0.0010 0.0003
MXNDArrayGetShape 2630 1.2210 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1116 9.7940 0.0070 0.0280 0.0088
MXNDArrayFree 12434 127.2420 0.0000 0.9140 0.0102
MXNet C API Calls 55546 542.9030 487.3580 542.9030 27.7725
MXNet C API Concurrency 111091 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 956 0.4590 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4780 2.4910 0.0000 0.0230 0.0005
MXNDArraySetGradState 4780 2.4370 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 956 0.4150 0.0000 0.0010 0.0004
MXAutogradBackwardEx 478 259.9130 0.5250 0.6810 0.5438
MXNDArrayGetContext 7732 3.3360 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16098 409.1020 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1912 4649.9570 0.7010 3.7500 2.4320
FullyConnected 3348 316.4840 0.0080 0.3430 0.0945
argmax 1116 21.7790 0.0050 0.0480 0.0195
_backward_pick 956 14.9580 0.0130 0.0260 0.0156
_backward_mul_scalar 956 4.1200 0.0030 0.0160 0.0043
DeleteVariable 41120 292.4930 0.0010 0.9010 0.0071
Flatten 1116 46.1070 0.0080 0.1120 0.0413
Activation 4464 202.0210 0.0030 0.3430 0.0453
SetValueOp 956 6.1050 0.0050 0.0180 0.0064
pick 956 7.4570 0.0060 0.0280 0.0078
Pooling 2232 3075.9619 0.0640 3.1720 1.3781
Convolution 2232 12665.0479 0.2260 13.2050 5.6743
CopyCPU2CPU 1116 6.3920 0.0030 0.0340 0.0057
log_softmax 956 29.3990 0.0150 0.0500 0.0308
_backward_copy 956 49.0890 0.0220 0.0760 0.0513
_backward_Convolution 1912 17611.4609 2.6370 12.5610 9.2110
_mul_scalar 956 3.3730 0.0020 0.0230 0.0035
_backward_log_softmax 956 15.9580 0.0090 0.0400 0.0167
mean 3028 57.8130 0.0080 0.0730 0.0191
_backward_mean 956 10.4030 0.0080 0.1030 0.0109
_backward_FullyConnected 2868 534.3260 0.0280 0.5940 0.1863
WaitForVar 2072 18.3790 0.0060 0.0280 0.0089
broadcast_equal 1116 5.5190 0.0030 0.0100 0.0049
sgd_update 9560 92.3800 0.0020 0.0640 0.0097
_backward_Activation 3824 279.7460 0.0070 0.3900 0.0732
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37846 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3354 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 478 1902.3521 0.3060 73.5850 3.9798
MXNDArrayCreateEx 559 19.1330 0.0310 0.0640 0.0342
MXNDArrayWaitAll 479 10350.1650 5.6130 24.8060 21.6079
MXNDArraySyncCopyToCPU 1038 8203.0879 0.1180 24.1980 7.9028
MXNDArrayGetDType 1038 0.3630 0.0000 0.0010 0.0003
MXNDArrayGetShape 2635 1.2230 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1118 9.8120 0.0070 0.0280 0.0088
MXNDArrayFree 12457 127.4740 0.0000 0.9140 0.0102
MXNet C API Calls 55653 543.0100 487.3580 543.0100 27.8260
MXNet C API Concurrency 111305 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 958 0.4600 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4790 2.4980 0.0000 0.0230 0.0005
MXNDArraySetGradState 4790 2.4430 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 958 0.4160 0.0000 0.0010 0.0004
MXAutogradBackwardEx 479 260.4790 0.5250 0.6810 0.5438
MXNDArrayGetContext 7746 3.3410 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16129 409.8880 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1916 4659.7612 0.7010 3.7500 2.4320
FullyConnected 3354 317.0450 0.0080 0.3430 0.0945
argmax 1118 21.8180 0.0050 0.0480 0.0195
_backward_pick 958 14.9910 0.0130 0.0260 0.0156
_backward_mul_scalar 958 4.1290 0.0030 0.0160 0.0043
DeleteVariable 41200 293.0230 0.0010 0.9010 0.0071
Flatten 1118 46.2000 0.0080 0.1120 0.0413
Activation 4472 202.3860 0.0030 0.3430 0.0453
SetValueOp 958 6.1190 0.0050 0.0180 0.0064
pick 958 7.4730 0.0060 0.0280 0.0078
Pooling 2236 3081.3879 0.0640 3.1720 1.3781
Convolution 2236 12687.3555 0.2260 13.2050 5.6741
CopyCPU2CPU 1118 6.4010 0.0030 0.0340 0.0057
log_softmax 958 29.4600 0.0150 0.0500 0.0308
_backward_copy 958 49.2000 0.0220 0.0760 0.0514
_backward_Convolution 1916 17648.3145 2.6370 12.5610 9.2110
_mul_scalar 958 3.3810 0.0020 0.0230 0.0035
_backward_log_softmax 958 15.9900 0.0090 0.0400 0.0167
mean 3034 57.9320 0.0080 0.0730 0.0191
_backward_mean 958 10.4250 0.0080 0.1030 0.0109
_backward_FullyConnected 2874 535.4190 0.0280 0.5940 0.1863
WaitForVar 2076 18.4160 0.0060 0.0280 0.0089
broadcast_equal 1118 5.5300 0.0030 0.0100 0.0049
sgd_update 9580 92.5870 0.0020 0.0640 0.0097
_backward_Activation 3832 280.2830 0.0070 0.3900 0.0731
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37920 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3360 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 479 1906.2570 0.3060 73.5850 3.9797
MXNDArrayCreateEx 560 19.1670 0.0310 0.0640 0.0342
MXNDArrayWaitAll 480 10371.8340 5.6130 24.8060 21.6080
MXNDArraySyncCopyToCPU 1040 8217.5303 0.1180 24.1980 7.9015
MXNDArrayGetDType 1040 0.3640 0.0000 0.0010 0.0003
MXNDArrayGetShape 2640 1.2240 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1120 9.8290 0.0070 0.0280 0.0088
MXNDArrayFree 12480 127.6990 0.0000 0.9140 0.0102
MXNet C API Calls 55760 543.1170 487.3580 543.1170 27.8795
MXNet C API Concurrency 111519 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 960 0.4600 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4800 2.5020 0.0000 0.0230 0.0005
MXNDArraySetGradState 4800 2.4470 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 960 0.4160 0.0000 0.0010 0.0004
MXAutogradBackwardEx 480 261.0160 0.5250 0.6810 0.5438
MXNDArrayGetContext 7760 3.3460 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16160 410.6850 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1920 4669.3711 0.7010 3.7500 2.4320
FullyConnected 3360 317.6020 0.0080 0.3430 0.0945
argmax 1120 21.8560 0.0050 0.0480 0.0195
_backward_pick 960 15.0230 0.0130 0.0260 0.0156
_backward_mul_scalar 960 4.1380 0.0030 0.0160 0.0043
DeleteVariable 41280 293.5830 0.0010 0.9010 0.0071
Flatten 1120 46.2860 0.0080 0.1120 0.0413
Activation 4480 202.7440 0.0030 0.3430 0.0453
SetValueOp 960 6.1330 0.0050 0.0180 0.0064
pick 960 7.4880 0.0060 0.0280 0.0078
Pooling 2240 3086.7600 0.0640 3.1720 1.3780
Convolution 2240 12709.5234 0.2260 13.2050 5.6739
CopyCPU2CPU 1120 6.4120 0.0030 0.0340 0.0057
log_softmax 960 29.5200 0.0150 0.0500 0.0308
_backward_copy 960 49.3140 0.0220 0.0760 0.0514
_backward_Convolution 1920 17685.1074 2.6370 12.5610 9.2110
_mul_scalar 960 3.3880 0.0020 0.0230 0.0035
_backward_log_softmax 960 16.0210 0.0090 0.0400 0.0167
mean 3040 58.0530 0.0080 0.0730 0.0191
_backward_mean 960 10.4460 0.0080 0.1030 0.0109
_backward_FullyConnected 2880 536.5060 0.0280 0.5940 0.1863
WaitForVar 2080 18.4530 0.0060 0.0280 0.0089
broadcast_equal 1120 5.5400 0.0030 0.0100 0.0049
sgd_update 9600 92.7970 0.0020 0.0640 0.0097
_backward_Activation 3840 280.8360 0.0070 0.3900 0.0731
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 37994 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3366 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 480 1910.1770 0.3060 73.5850 3.9795
MXNDArrayCreateEx 561 19.2010 0.0310 0.0640 0.0342
MXNDArrayWaitAll 481 10393.6709 5.6130 24.8060 21.6085
MXNDArraySyncCopyToCPU 1042 8231.8750 0.1180 24.1980 7.9001
MXNDArrayGetDType 1042 0.3650 0.0000 0.0010 0.0004
MXNDArrayGetShape 2645 1.2250 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1122 9.8470 0.0070 0.0280 0.0088
MXNDArrayFree 12503 127.9390 0.0000 0.9140 0.0102
MXNet C API Calls 55867 543.2240 487.3580 543.2240 27.9330
MXNet C API Concurrency 111733 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 962 0.4620 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4810 2.5070 0.0000 0.0230 0.0005
MXNDArraySetGradState 4810 2.4530 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 962 0.4160 0.0000 0.0010 0.0004
MXAutogradBackwardEx 481 261.5740 0.5250 0.6810 0.5438
MXNDArrayGetContext 7774 3.3520 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16191 411.5010 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1924 4679.0889 0.7010 3.7500 2.4320
FullyConnected 3366 318.2210 0.0080 0.3430 0.0945
argmax 1122 21.8960 0.0050 0.0480 0.0195
_backward_pick 962 15.0560 0.0130 0.0260 0.0157
_backward_mul_scalar 962 4.1470 0.0030 0.0160 0.0043
DeleteVariable 41360 294.1200 0.0010 0.9010 0.0071
Flatten 1122 46.3730 0.0080 0.1120 0.0413
Activation 4488 203.1050 0.0030 0.3430 0.0453
SetValueOp 962 6.1460 0.0050 0.0180 0.0064
pick 962 7.5030 0.0060 0.0280 0.0078
Pooling 2244 3092.0959 0.0640 3.1720 1.3779
Convolution 2244 12731.8047 0.2260 13.2050 5.6737
CopyCPU2CPU 1122 6.4220 0.0030 0.0340 0.0057
log_softmax 962 29.5810 0.0150 0.0500 0.0307
_backward_copy 962 49.4270 0.0220 0.0760 0.0514
_backward_Convolution 1924 17722.1211 2.6370 12.5610 9.2111
_mul_scalar 962 3.3960 0.0020 0.0230 0.0035
_backward_log_softmax 962 16.0530 0.0090 0.0400 0.0167
mean 3046 58.1730 0.0080 0.0730 0.0191
_backward_mean 962 10.4670 0.0080 0.1030 0.0109
_backward_FullyConnected 2886 537.5930 0.0280 0.5940 0.1863
WaitForVar 2084 18.4910 0.0060 0.0280 0.0089
broadcast_equal 1122 5.5490 0.0030 0.0100 0.0049
sgd_update 9620 92.9530 0.0020 0.0640 0.0097
_backward_Activation 3848 281.4720 0.0070 0.3900 0.0731
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38068 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3372 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 481 1914.0790 0.3060 73.5850 3.9794
MXNDArrayCreateEx 562 19.2350 0.0310 0.0640 0.0342
MXNDArrayWaitAll 482 10415.5293 5.6130 24.8060 21.6090
MXNDArraySyncCopyToCPU 1044 8246.8564 0.1180 24.1980 7.8993
MXNDArrayGetDType 1044 0.3660 0.0000 0.0010 0.0004
MXNDArrayGetShape 2650 1.2290 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1124 9.8640 0.0070 0.0280 0.0088
MXNDArrayFree 12526 128.1680 0.0000 0.9140 0.0102
MXNet C API Calls 55974 543.3310 487.3580 543.3310 27.9865
MXNet C API Concurrency 111947 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 964 0.4630 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4820 2.5110 0.0000 0.0230 0.0005
MXNDArraySetGradState 4820 2.4600 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 964 0.4160 0.0000 0.0010 0.0004
MXAutogradBackwardEx 482 262.1100 0.5250 0.6810 0.5438
MXNDArrayGetContext 7788 3.3570 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16222 412.2890 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1928 4688.8159 0.7010 3.7500 2.4320
FullyConnected 3372 319.0140 0.0080 0.3430 0.0946
argmax 1124 21.9410 0.0050 0.0480 0.0195
_backward_pick 964 15.0910 0.0130 0.0260 0.0157
_backward_mul_scalar 964 4.1560 0.0030 0.0160 0.0043
DeleteVariable 41440 294.6690 0.0010 0.9010 0.0071
Flatten 1124 46.4600 0.0080 0.1120 0.0413
Activation 4496 203.5060 0.0030 0.3430 0.0453
SetValueOp 964 6.1600 0.0050 0.0180 0.0064
pick 964 7.5190 0.0060 0.0280 0.0078
Pooling 2248 3097.4719 0.0640 3.1720 1.3779
Convolution 2248 12754.3271 0.2260 13.2050 5.6736
CopyCPU2CPU 1124 6.4320 0.0030 0.0340 0.0057
log_softmax 964 29.6530 0.0150 0.0500 0.0308
_backward_copy 964 49.5370 0.0220 0.0760 0.0514
_backward_Convolution 1928 17759.1758 2.6370 12.5610 9.2112
_mul_scalar 964 3.4040 0.0020 0.0230 0.0035
_backward_log_softmax 964 16.0950 0.0090 0.0400 0.0167
mean 3052 58.2850 0.0080 0.0730 0.0191
_backward_mean 964 10.4890 0.0080 0.1030 0.0109
_backward_FullyConnected 2892 539.0180 0.0280 0.5940 0.1864
WaitForVar 2088 18.5300 0.0060 0.0280 0.0089
broadcast_equal 1124 5.5600 0.0030 0.0100 0.0049
sgd_update 9640 93.2200 0.0020 0.0640 0.0097
_backward_Activation 3856 282.0720 0.0070 0.3900 0.0732
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38142 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3378 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 482 1917.9919 0.3060 73.5850 3.9792
MXNDArrayCreateEx 563 19.2690 0.0310 0.0640 0.0342
MXNDArrayWaitAll 483 10437.2930 5.6130 24.8060 21.6093
MXNDArraySyncCopyToCPU 1046 8261.2256 0.1180 24.1980 7.8979
MXNDArrayGetDType 1046 0.3670 0.0000 0.0010 0.0004
MXNDArrayGetShape 2655 1.2300 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1126 9.8810 0.0070 0.0280 0.0088
MXNDArrayFree 12549 128.3930 0.0000 0.9140 0.0102
MXNet C API Calls 56081 543.4380 487.3580 543.4380 28.0400
MXNet C API Concurrency 112161 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 966 0.4630 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4830 2.5160 0.0000 0.0230 0.0005
MXNDArraySetGradState 4830 2.4640 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 966 0.4160 0.0000 0.0010 0.0004
MXAutogradBackwardEx 483 262.6450 0.5250 0.6810 0.5438
MXNDArrayGetContext 7802 3.3660 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16253 413.0840 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1932 4698.5371 0.7010 3.7500 2.4320
FullyConnected 3378 319.5440 0.0080 0.3430 0.0946
argmax 1126 21.9790 0.0050 0.0480 0.0195
_backward_pick 966 15.1220 0.0130 0.0260 0.0157
_backward_mul_scalar 966 4.1650 0.0030 0.0160 0.0043
DeleteVariable 41520 295.1960 0.0010 0.9010 0.0071
Flatten 1126 46.5470 0.0080 0.1120 0.0413
Activation 4504 203.9200 0.0030 0.3430 0.0453
SetValueOp 966 6.1730 0.0050 0.0180 0.0064
pick 966 7.5340 0.0060 0.0280 0.0078
Pooling 2252 3102.8220 0.0640 3.1720 1.3778
Convolution 2252 12776.6396 0.2260 13.2050 5.6735
CopyCPU2CPU 1126 6.4420 0.0030 0.0340 0.0057
log_softmax 966 29.7140 0.0150 0.0500 0.0308
_backward_copy 966 49.6510 0.0220 0.0760 0.0514
_backward_Convolution 1932 17795.9961 2.6370 12.5610 9.2112
_mul_scalar 966 3.4110 0.0020 0.0230 0.0035
_backward_log_softmax 966 16.1260 0.0090 0.0400 0.0167
mean 3058 58.4010 0.0080 0.0730 0.0191
_backward_mean 966 10.5100 0.0080 0.1030 0.0109
_backward_FullyConnected 2898 540.1030 0.0280 0.5940 0.1864
WaitForVar 2092 18.5670 0.0060 0.0280 0.0089
broadcast_equal 1126 5.5710 0.0030 0.0100 0.0049
sgd_update 9660 93.4250 0.0020 0.0640 0.0097
_backward_Activation 3864 282.6410 0.0070 0.3900 0.0731
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38216 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3384 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 483 1921.8950 0.3060 73.5850 3.9791
MXNDArrayCreateEx 564 19.3040 0.0310 0.0640 0.0342
MXNDArrayWaitAll 484 10459.0283 5.6130 24.8060 21.6096
MXNDArraySyncCopyToCPU 1048 8275.7188 0.1180 24.1980 7.8967
MXNDArrayGetDType 1048 0.3680 0.0000 0.0010 0.0004
MXNDArrayGetShape 2660 1.2310 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1128 9.8990 0.0070 0.0280 0.0088
MXNDArrayFree 12572 128.6200 0.0000 0.9140 0.0102
MXNet C API Calls 56188 543.5450 487.3580 543.5450 28.0935
MXNet C API Concurrency 112375 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 968 0.4650 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4840 2.5200 0.0000 0.0230 0.0005
MXNDArraySetGradState 4840 2.4710 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 968 0.4170 0.0000 0.0010 0.0004
MXAutogradBackwardEx 484 263.1790 0.5250 0.6810 0.5438
MXNDArrayGetContext 7816 3.3720 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16284 413.8820 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1936 4708.2778 0.7010 3.7500 2.4320
FullyConnected 3384 320.1470 0.0080 0.3430 0.0946
argmax 1128 22.0170 0.0050 0.0480 0.0195
_backward_pick 968 15.1530 0.0130 0.0260 0.0157
_backward_mul_scalar 968 4.1730 0.0030 0.0160 0.0043
DeleteVariable 41600 295.7170 0.0010 0.9010 0.0071
Flatten 1128 46.6310 0.0080 0.1120 0.0413
Activation 4512 204.2810 0.0030 0.3430 0.0453
SetValueOp 968 6.1870 0.0050 0.0180 0.0064
pick 968 7.5490 0.0060 0.0280 0.0078
Pooling 2256 3108.2151 0.0640 3.1720 1.3778
Convolution 2256 12798.8477 0.2260 13.2050 5.6732
CopyCPU2CPU 1128 6.4530 0.0030 0.0340 0.0057
log_softmax 968 29.7740 0.0150 0.0500 0.0308
_backward_copy 968 49.7640 0.0220 0.0760 0.0514
_backward_Convolution 1936 17832.8418 2.6370 12.5610 9.2112
_mul_scalar 968 3.4180 0.0020 0.0230 0.0035
_backward_log_softmax 968 16.1580 0.0090 0.0400 0.0167
mean 3064 58.5210 0.0080 0.0730 0.0191
_backward_mean 968 10.5320 0.0080 0.1030 0.0109
_backward_FullyConnected 2904 541.1900 0.0280 0.5940 0.1864
WaitForVar 2096 18.6030 0.0060 0.0280 0.0089
broadcast_equal 1128 5.5810 0.0030 0.0100 0.0049
sgd_update 9680 93.6340 0.0020 0.0640 0.0097
_backward_Activation 3872 283.2130 0.0070 0.3900 0.0731
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38290 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3390 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 484 1925.7920 0.3060 73.5850 3.9789
MXNDArrayCreateEx 565 19.3380 0.0310 0.0640 0.0342
MXNDArrayWaitAll 485 10470.7949 5.6130 24.8060 21.5893
MXNDArraySyncCopyToCPU 1050 8299.9111 0.1180 24.1980 7.9047
MXNDArrayGetDType 1050 0.3680 0.0000 0.0010 0.0004
MXNDArrayGetShape 2665 1.2320 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1130 9.9170 0.0070 0.0280 0.0088
MXNDArrayFree 12595 128.8600 0.0000 0.9140 0.0102
MXNet C API Calls 56295 543.6520 487.3580 543.6520 28.1470
MXNet C API Concurrency 112589 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 970 0.4650 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4850 2.5250 0.0000 0.0230 0.0005
MXNDArraySetGradState 4850 2.4740 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 970 0.4180 0.0000 0.0010 0.0004
MXAutogradBackwardEx 485 263.7130 0.5250 0.6810 0.5437
MXNDArrayGetContext 7830 3.3780 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16315 414.6800 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1940 4718.0620 0.7010 3.7500 2.4320
FullyConnected 3390 320.7030 0.0080 0.3430 0.0946
argmax 1130 22.0560 0.0050 0.0480 0.0195
_backward_pick 970 15.1850 0.0130 0.0260 0.0157
_backward_mul_scalar 970 4.1820 0.0030 0.0160 0.0043
DeleteVariable 41680 296.2610 0.0010 0.9010 0.0071
Flatten 1130 46.7220 0.0080 0.1120 0.0413
Activation 4520 204.6430 0.0030 0.3430 0.0453
SetValueOp 970 6.2000 0.0050 0.0180 0.0064
pick 970 7.5640 0.0060 0.0280 0.0078
Pooling 2260 3113.5430 0.0640 3.1720 1.3777
Convolution 2260 12821.0938 0.2260 13.2050 5.6731
CopyCPU2CPU 1130 6.5230 0.0030 0.0360 0.0058
log_softmax 970 29.8340 0.0150 0.0500 0.0308
_backward_copy 970 49.8770 0.0220 0.0760 0.0514
_backward_Convolution 1940 17869.8516 2.6370 12.5610 9.2113
_mul_scalar 970 3.4260 0.0020 0.0230 0.0035
_backward_log_softmax 970 16.1890 0.0090 0.0400 0.0167
mean 3070 58.5980 0.0080 0.0730 0.0191
_backward_mean 970 10.5540 0.0080 0.1030 0.0109
_backward_FullyConnected 2910 542.2780 0.0280 0.5940 0.1863
WaitForVar 2100 18.6390 0.0060 0.0280 0.0089
broadcast_equal 1130 5.5920 0.0030 0.0100 0.0049
sgd_update 9700 93.7860 0.0020 0.0640 0.0097
_backward_Activation 3880 284.1110 0.0070 0.3900 0.0732
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38364 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3396 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 485 1929.6980 0.3060 73.5850 3.9788
MXNDArrayCreateEx 566 19.3720 0.0310 0.0640 0.0342
MXNDArrayWaitAll 486 10492.6426 5.6130 24.8060 21.5898
MXNDArraySyncCopyToCPU 1052 8314.8994 0.1180 24.1980 7.9039
MXNDArrayGetDType 1052 0.3690 0.0000 0.0010 0.0004
MXNDArrayGetShape 2670 1.2350 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1132 9.9340 0.0070 0.0280 0.0088
MXNDArrayFree 12618 129.0980 0.0000 0.9140 0.0102
MXNet C API Calls 56402 543.7590 487.3580 543.7590 28.2005
MXNet C API Concurrency 112803 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 972 0.4650 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4860 2.5310 0.0000 0.0230 0.0005
MXNDArraySetGradState 4860 2.4780 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 972 0.4180 0.0000 0.0010 0.0004
MXAutogradBackwardEx 486 264.2500 0.5250 0.6810 0.5437
MXNDArrayGetContext 7844 3.3810 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16346 415.4770 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1944 4727.7148 0.7010 3.7500 2.4320
FullyConnected 3396 321.4870 0.0080 0.3430 0.0947
argmax 1132 22.1000 0.0050 0.0480 0.0195
_backward_pick 972 15.2200 0.0130 0.0260 0.0157
_backward_mul_scalar 972 4.1920 0.0030 0.0160 0.0043
DeleteVariable 41760 296.8000 0.0010 0.9010 0.0071
Flatten 1132 46.8110 0.0080 0.1120 0.0414
Activation 4528 205.0300 0.0030 0.3430 0.0453
SetValueOp 972 6.2140 0.0050 0.0180 0.0064
pick 972 7.5820 0.0060 0.0280 0.0078
Pooling 2264 3118.9360 0.0640 3.1720 1.3776
Convolution 2264 12843.6084 0.2260 13.2050 5.6730
CopyCPU2CPU 1132 6.5330 0.0030 0.0360 0.0058
log_softmax 972 29.9070 0.0150 0.0500 0.0308
_backward_copy 972 49.9880 0.0220 0.0760 0.0514
_backward_Convolution 1944 17906.9395 2.6370 12.5610 9.2114
_mul_scalar 972 3.4330 0.0020 0.0230 0.0035
_backward_log_softmax 972 16.2300 0.0090 0.0400 0.0167
mean 3076 58.7090 0.0080 0.0730 0.0191
_backward_mean 972 10.5770 0.0080 0.1030 0.0109
_backward_FullyConnected 2916 543.7120 0.0280 0.5940 0.1865
WaitForVar 2104 18.6760 0.0060 0.0280 0.0089
broadcast_equal 1132 5.6020 0.0030 0.0100 0.0049
sgd_update 9720 94.0670 0.0020 0.0640 0.0097
_backward_Activation 3888 284.7250 0.0070 0.3900 0.0732
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38438 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3402 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 486 1931.6639 0.3060 73.5850 3.9746
MXNDArrayCreateEx 567 19.4050 0.0310 0.0640 0.0342
MXNDArrayWaitAll 487 10514.2939 5.6130 24.8060 21.5899
MXNDArraySyncCopyToCPU 1054 8329.3926 0.1180 24.1980 7.9026
MXNDArrayGetDType 1054 0.3700 0.0000 0.0010 0.0004
MXNDArrayGetShape 2675 1.2370 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1134 9.9510 0.0070 0.0280 0.0088
MXNDArrayFree 12641 129.3240 0.0000 0.9140 0.0102
MXNet C API Calls 56509 543.8660 487.3580 543.8660 28.2540
MXNet C API Concurrency 113017 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 974 0.4660 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4870 2.5360 0.0000 0.0230 0.0005
MXNDArraySetGradState 4870 2.4830 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 974 0.4180 0.0000 0.0010 0.0004
MXAutogradBackwardEx 487 264.7880 0.5250 0.6810 0.5437
MXNDArrayGetContext 7858 3.3890 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16377 416.2840 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1948 4737.3232 0.7010 3.7500 2.4319
FullyConnected 3402 322.0410 0.0080 0.3430 0.0947
argmax 1134 22.1380 0.0050 0.0480 0.0195
_backward_pick 974 15.2510 0.0130 0.0260 0.0157
_backward_mul_scalar 974 4.2010 0.0030 0.0160 0.0043
DeleteVariable 41840 297.3300 0.0010 0.9010 0.0071
Flatten 1134 46.8940 0.0080 0.1120 0.0414
Activation 4536 205.3920 0.0030 0.3430 0.0453
SetValueOp 974 6.2290 0.0050 0.0180 0.0064
pick 974 7.5980 0.0060 0.0280 0.0078
Pooling 2268 3124.3560 0.0640 3.1720 1.3776
Convolution 2268 12865.8574 0.2260 13.2050 5.6728
CopyCPU2CPU 1134 6.5430 0.0030 0.0360 0.0058
log_softmax 974 29.9670 0.0150 0.0500 0.0308
_backward_copy 974 50.1010 0.0220 0.0760 0.0514
_backward_Convolution 1948 17943.6855 2.6370 12.5610 9.2113
_mul_scalar 974 3.4400 0.0020 0.0230 0.0035
_backward_log_softmax 974 16.2620 0.0090 0.0400 0.0167
mean 3082 58.8280 0.0080 0.0730 0.0191
_backward_mean 974 10.5990 0.0080 0.1030 0.0109
_backward_FullyConnected 2922 544.7960 0.0280 0.5940 0.1864
WaitForVar 2108 18.7130 0.0060 0.0280 0.0089
broadcast_equal 1134 5.6130 0.0030 0.0100 0.0049
sgd_update 9740 94.2190 0.0020 0.0640 0.0097
_backward_Activation 3896 285.3640 0.0070 0.3900 0.0732
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38512 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3408 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 487 1935.5601 0.3060 73.5850 3.9745
MXNDArrayCreateEx 568 19.4390 0.0310 0.0640 0.0342
MXNDArrayWaitAll 488 10536.0762 5.6130 24.8060 21.5903
MXNDArraySyncCopyToCPU 1056 8343.8271 0.1180 24.1980 7.9014
MXNDArrayGetDType 1056 0.3710 0.0000 0.0010 0.0004
MXNDArrayGetShape 2680 1.2380 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1136 9.9670 0.0070 0.0280 0.0088
MXNDArrayFree 12664 129.5480 0.0000 0.9140 0.0102
MXNet C API Calls 56616 543.9730 487.3580 543.9730 28.3075
MXNet C API Concurrency 113231 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 976 0.4670 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4880 2.5420 0.0000 0.0230 0.0005
MXNDArraySetGradState 4880 2.4880 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 976 0.4180 0.0000 0.0010 0.0004
MXAutogradBackwardEx 488 265.3210 0.5250 0.6810 0.5437
MXNDArrayGetContext 7872 3.3930 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16408 417.0600 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1952 4747.1182 0.7010 3.7500 2.4319
FullyConnected 3408 322.6290 0.0080 0.3430 0.0947
argmax 1136 22.1760 0.0050 0.0480 0.0195
_backward_pick 976 15.2840 0.0130 0.0260 0.0157
_backward_mul_scalar 976 4.2100 0.0030 0.0160 0.0043
DeleteVariable 41920 297.8400 0.0010 0.9010 0.0071
Flatten 1136 46.9790 0.0080 0.1120 0.0414
Activation 4544 205.7510 0.0030 0.3430 0.0453
SetValueOp 976 6.2420 0.0050 0.0180 0.0064
pick 976 7.6140 0.0060 0.0280 0.0078
Pooling 2272 3129.7129 0.0640 3.1720 1.3775
Convolution 2272 12888.0293 0.2260 13.2050 5.6725
CopyCPU2CPU 1136 6.5530 0.0030 0.0360 0.0058
log_softmax 976 30.0260 0.0150 0.0500 0.0308
_backward_copy 976 50.2120 0.0220 0.0760 0.0514
_backward_Convolution 1952 17980.4844 2.6370 12.5610 9.2113
_mul_scalar 976 3.4470 0.0020 0.0230 0.0035
_backward_log_softmax 976 16.2940 0.0090 0.0400 0.0167
mean 3088 58.9450 0.0080 0.0730 0.0191
_backward_mean 976 10.6220 0.0080 0.1030 0.0109
_backward_FullyConnected 2928 545.8840 0.0280 0.5940 0.1864
WaitForVar 2112 18.7490 0.0060 0.0280 0.0089
broadcast_equal 1136 5.6240 0.0030 0.0100 0.0050
sgd_update 9760 94.3720 0.0020 0.0640 0.0097
_backward_Activation 3904 286.0140 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38586 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3414 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 488 1939.4640 0.3060 73.5850 3.9743
MXNDArrayCreateEx 569 19.4710 0.0310 0.0640 0.0342
MXNDArrayWaitAll 489 10557.9297 5.6130 24.8060 21.5909
MXNDArraySyncCopyToCPU 1058 8358.2988 0.1180 24.1980 7.9001
MXNDArrayGetDType 1058 0.3720 0.0000 0.0010 0.0004
MXNDArrayGetShape 2685 1.2410 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1138 9.9860 0.0070 0.0280 0.0088
MXNDArrayFree 12687 129.7750 0.0000 0.9140 0.0102
MXNet C API Calls 56723 544.0800 487.3580 544.0800 28.3610
MXNet C API Concurrency 113445 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 978 0.4680 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4890 2.5450 0.0000 0.0230 0.0005
MXNDArraySetGradState 4890 2.4960 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 978 0.4190 0.0000 0.0010 0.0004
MXAutogradBackwardEx 489 265.8550 0.5250 0.6810 0.5437
MXNDArrayGetContext 7886 3.4010 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16439 417.8400 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1956 4756.8281 0.7010 3.7500 2.4319
FullyConnected 3414 323.1590 0.0080 0.3430 0.0947
argmax 1138 22.2150 0.0050 0.0480 0.0195
_backward_pick 978 15.3150 0.0130 0.0260 0.0157
_backward_mul_scalar 978 4.2190 0.0030 0.0160 0.0043
DeleteVariable 42000 298.3670 0.0010 0.9010 0.0071
Flatten 1138 47.0640 0.0080 0.1120 0.0414
Activation 4552 206.1120 0.0030 0.3430 0.0453
SetValueOp 978 6.2540 0.0050 0.0180 0.0064
pick 978 7.6290 0.0060 0.0280 0.0078
Pooling 2276 3135.1350 0.0640 3.1720 1.3775
Convolution 2276 12910.2783 0.2260 13.2050 5.6724
CopyCPU2CPU 1138 6.5620 0.0030 0.0360 0.0058
log_softmax 978 30.0860 0.0150 0.0500 0.0308
_backward_copy 978 50.3250 0.0220 0.0760 0.0515
_backward_Convolution 1956 18017.5312 2.6370 12.5610 9.2114
_mul_scalar 978 3.4540 0.0020 0.0230 0.0035
_backward_log_softmax 978 16.3260 0.0090 0.0400 0.0167
mean 3094 59.0630 0.0080 0.0730 0.0191
_backward_mean 978 10.6420 0.0080 0.1030 0.0109
_backward_FullyConnected 2934 546.9700 0.0280 0.5940 0.1864
WaitForVar 2116 18.7860 0.0060 0.0280 0.0089
broadcast_equal 1138 5.6340 0.0030 0.0100 0.0050
sgd_update 9780 94.5850 0.0020 0.0640 0.0097
_backward_Activation 3912 286.5720 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38660 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3420 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 489 1943.3680 0.3060 73.5850 3.9742
MXNDArrayCreateEx 570 19.5040 0.0310 0.0640 0.0342
MXNDArrayWaitAll 490 10579.7861 5.6130 24.8060 21.5914
MXNDArraySyncCopyToCPU 1060 8373.2676 0.1180 24.1980 7.8993
MXNDArrayGetDType 1060 0.3720 0.0000 0.0010 0.0004
MXNDArrayGetShape 2690 1.2440 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1140 10.0040 0.0070 0.0280 0.0088
MXNDArrayFree 12710 130.0060 0.0000 0.9140 0.0102
MXNet C API Calls 56830 544.1870 487.3580 544.1870 28.4145
MXNet C API Concurrency 113659 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 980 0.4690 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4900 2.5510 0.0000 0.0230 0.0005
MXNDArraySetGradState 4900 2.5010 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 980 0.4200 0.0000 0.0010 0.0004
MXAutogradBackwardEx 490 266.3850 0.5250 0.6810 0.5436
MXNDArrayGetContext 7900 3.4060 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16470 418.6120 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1960 4766.5991 0.7010 3.7500 2.4319
FullyConnected 3420 323.9640 0.0080 0.3430 0.0947
argmax 1140 22.2600 0.0050 0.0480 0.0195
_backward_pick 980 15.3500 0.0130 0.0260 0.0157
_backward_mul_scalar 980 4.2270 0.0030 0.0160 0.0043
DeleteVariable 42080 298.9170 0.0010 0.9010 0.0071
Flatten 1140 47.1490 0.0080 0.1120 0.0414
Activation 4560 206.5090 0.0030 0.3430 0.0453
SetValueOp 980 6.2670 0.0050 0.0180 0.0064
pick 980 7.6460 0.0060 0.0280 0.0078
Pooling 2280 3140.4929 0.0640 3.1720 1.3774
Convolution 2280 12932.7236 0.2260 13.2050 5.6722
CopyCPU2CPU 1140 6.5720 0.0030 0.0360 0.0058
log_softmax 980 30.1600 0.0150 0.0500 0.0308
_backward_copy 980 50.4340 0.0220 0.0760 0.0515
_backward_Convolution 1960 18054.5918 2.6370 12.5610 9.2115
_mul_scalar 980 3.4610 0.0020 0.0230 0.0035
_backward_log_softmax 980 16.3680 0.0090 0.0400 0.0167
mean 3100 59.1710 0.0080 0.0730 0.0191
_backward_mean 980 10.6650 0.0080 0.1030 0.0109
_backward_FullyConnected 2940 548.4020 0.0280 0.5940 0.1865
WaitForVar 2120 18.8250 0.0060 0.0280 0.0089
broadcast_equal 1140 5.6450 0.0030 0.0100 0.0050
sgd_update 9800 94.8030 0.0020 0.0640 0.0097
_backward_Activation 3920 287.2520 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38734 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3426 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 490 1947.2820 0.3060 73.5850 3.9740
MXNDArrayCreateEx 571 19.5370 0.0310 0.0640 0.0342
MXNDArrayWaitAll 491 10601.4961 5.6130 24.8060 21.5916
MXNDArraySyncCopyToCPU 1062 8387.7236 0.1180 24.1980 7.8980
MXNDArrayGetDType 1062 0.3730 0.0000 0.0010 0.0004
MXNDArrayGetShape 2695 1.2470 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1142 10.0220 0.0070 0.0280 0.0088
MXNDArrayFree 12733 130.2380 0.0000 0.9140 0.0102
MXNet C API Calls 56937 544.2940 487.3580 544.2940 28.4680
MXNet C API Concurrency 113873 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 982 0.4700 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4910 2.5550 0.0000 0.0230 0.0005
MXNDArraySetGradState 4910 2.5050 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 982 0.4200 0.0000 0.0010 0.0004
MXAutogradBackwardEx 491 266.9220 0.5250 0.6810 0.5436
MXNDArrayGetContext 7914 3.4110 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16501 419.4030 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1964 4776.3188 0.7010 3.7500 2.4319
FullyConnected 3426 324.4970 0.0080 0.3430 0.0947
argmax 1142 22.2990 0.0050 0.0480 0.0195
_backward_pick 982 15.3800 0.0130 0.0260 0.0157
_backward_mul_scalar 982 4.2360 0.0030 0.0160 0.0043
DeleteVariable 42160 299.4610 0.0010 0.9010 0.0071
Flatten 1142 47.2360 0.0080 0.1120 0.0414
Activation 4568 206.8700 0.0030 0.3430 0.0453
SetValueOp 982 6.2810 0.0050 0.0180 0.0064
pick 982 7.6620 0.0060 0.0280 0.0078
Pooling 2284 3145.9180 0.0640 3.1720 1.3774
Convolution 2284 12954.9961 0.2260 13.2050 5.6721
CopyCPU2CPU 1142 6.5820 0.0030 0.0360 0.0058
log_softmax 982 30.2200 0.0150 0.0500 0.0308
_backward_copy 982 50.5470 0.0220 0.0760 0.0515
_backward_Convolution 1964 18091.3457 2.6370 12.5610 9.2115
_mul_scalar 982 3.4690 0.0020 0.0230 0.0035
_backward_log_softmax 982 16.3990 0.0090 0.0400 0.0167
mean 3106 59.2870 0.0080 0.0730 0.0191
_backward_mean 982 10.6860 0.0080 0.1030 0.0109
_backward_FullyConnected 2946 549.4910 0.0280 0.5940 0.1865
WaitForVar 2124 18.8620 0.0060 0.0280 0.0089
broadcast_equal 1142 5.6540 0.0030 0.0100 0.0050
sgd_update 9820 95.0090 0.0020 0.0640 0.0097
_backward_Activation 3928 287.8110 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38808 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3432 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 491 1951.1860 0.3060 73.5850 3.9739
MXNDArrayCreateEx 572 19.5710 0.0310 0.0640 0.0342
MXNDArrayWaitAll 492 10623.2217 5.6130 24.8060 21.5919
MXNDArraySyncCopyToCPU 1064 8402.1279 0.1180 24.1980 7.8967
MXNDArrayGetDType 1064 0.3730 0.0000 0.0010 0.0004
MXNDArrayGetShape 2700 1.2490 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1144 10.0400 0.0070 0.0280 0.0088
MXNDArrayFree 12756 130.4610 0.0000 0.9140 0.0102
MXNet C API Calls 57044 544.4010 487.3580 544.4010 28.5215
MXNet C API Concurrency 114087 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 984 0.4710 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4920 2.5600 0.0000 0.0230 0.0005
MXNDArraySetGradState 4920 2.5080 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 984 0.4210 0.0000 0.0010 0.0004
MXAutogradBackwardEx 492 267.4530 0.5250 0.6810 0.5436
MXNDArrayGetContext 7928 3.4170 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16532 420.1790 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1968 4786.0508 0.7010 3.7500 2.4319
FullyConnected 3432 325.0230 0.0080 0.3430 0.0947
argmax 1144 22.3370 0.0050 0.0480 0.0195
_backward_pick 984 15.4110 0.0130 0.0260 0.0157
_backward_mul_scalar 984 4.2450 0.0030 0.0160 0.0043
DeleteVariable 42240 299.9980 0.0010 0.9010 0.0071
Flatten 1144 47.3200 0.0080 0.1120 0.0414
Activation 4576 207.2290 0.0030 0.3430 0.0453
SetValueOp 984 6.2940 0.0050 0.0180 0.0064
pick 984 7.6770 0.0060 0.0280 0.0078
Pooling 2288 3151.2639 0.0640 3.1720 1.3773
Convolution 2288 12977.1514 0.2260 13.2050 5.6718
CopyCPU2CPU 1144 6.5920 0.0030 0.0360 0.0058
log_softmax 984 30.2800 0.0150 0.0500 0.0308
_backward_copy 984 50.6590 0.0220 0.0760 0.0515
_backward_Convolution 1968 18128.1133 2.6370 12.5610 9.2114
_mul_scalar 984 3.4770 0.0020 0.0230 0.0035
_backward_log_softmax 984 16.4310 0.0090 0.0400 0.0167
mean 3112 59.3990 0.0080 0.0730 0.0191
_backward_mean 984 10.7070 0.0080 0.1030 0.0109
_backward_FullyConnected 2952 550.5840 0.0280 0.5940 0.1865
WaitForVar 2128 18.9000 0.0060 0.0280 0.0089
broadcast_equal 1144 5.6640 0.0030 0.0100 0.0050
sgd_update 9840 95.1610 0.0020 0.0640 0.0097
_backward_Activation 3936 288.4300 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38882 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3438 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 492 1955.2930 0.3060 73.5850 3.9742
MXNDArrayCreateEx 573 19.6050 0.0310 0.0640 0.0342
MXNDArrayWaitAll 493 10645.0938 5.6130 24.8060 21.5925
MXNDArraySyncCopyToCPU 1066 8416.6055 0.1180 24.1980 7.8955
MXNDArrayGetDType 1066 0.3740 0.0000 0.0010 0.0004
MXNDArrayGetShape 2705 1.2500 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1146 10.0570 0.0070 0.0280 0.0088
MXNDArrayFree 12779 130.6920 0.0000 0.9140 0.0102
MXNet C API Calls 57151 544.5080 487.3580 544.5080 28.5750
MXNet C API Concurrency 114301 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 986 0.4720 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4930 2.5670 0.0000 0.0230 0.0005
MXNDArraySetGradState 4930 2.5110 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 986 0.4220 0.0000 0.0010 0.0004
MXAutogradBackwardEx 493 268.0070 0.5250 0.6810 0.5436
MXNDArrayGetContext 7942 3.4230 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16563 420.9490 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1972 4795.8408 0.7010 3.7500 2.4320
FullyConnected 3438 325.5770 0.0080 0.3430 0.0947
argmax 1146 22.3750 0.0050 0.0480 0.0195
_backward_pick 986 15.4440 0.0130 0.0260 0.0157
_backward_mul_scalar 986 4.2540 0.0030 0.0160 0.0043
DeleteVariable 42320 300.5210 0.0010 0.9010 0.0071
Flatten 1146 47.4070 0.0080 0.1120 0.0414
Activation 4584 207.5890 0.0030 0.3430 0.0453
SetValueOp 986 6.3070 0.0050 0.0180 0.0064
pick 986 7.6930 0.0060 0.0280 0.0078
Pooling 2292 3156.6699 0.0640 3.1720 1.3773
Convolution 2292 12999.4053 0.2260 13.2050 5.6716
CopyCPU2CPU 1146 6.6010 0.0030 0.0360 0.0058
log_softmax 986 30.3410 0.0150 0.0500 0.0308
_backward_copy 986 50.7720 0.0220 0.0760 0.0515
_backward_Convolution 1972 18165.1289 2.6370 12.5610 9.2115
_mul_scalar 986 3.4840 0.0020 0.0230 0.0035
_backward_log_softmax 986 16.4620 0.0090 0.0400 0.0167
mean 3118 59.5210 0.0080 0.0730 0.0191
_backward_mean 986 10.7290 0.0080 0.1030 0.0109
_backward_FullyConnected 2958 551.6730 0.0280 0.5940 0.1865
WaitForVar 2132 18.9370 0.0060 0.0280 0.0089
broadcast_equal 1146 5.6750 0.0030 0.0100 0.0050
sgd_update 9860 95.3110 0.0020 0.0640 0.0097
_backward_Activation 3944 289.0480 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 38956 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3444 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 493 1959.1960 0.3060 73.5850 3.9740
MXNDArrayCreateEx 574 19.6390 0.0310 0.0640 0.0342
MXNDArrayWaitAll 494 10666.9648 5.6130 24.8060 21.5930
MXNDArraySyncCopyToCPU 1068 8431.5498 0.1180 24.1980 7.8947
MXNDArrayGetDType 1068 0.3740 0.0000 0.0010 0.0004
MXNDArrayGetShape 2710 1.2530 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1148 10.0740 0.0070 0.0280 0.0088
MXNDArrayFree 12802 130.9180 0.0000 0.9140 0.0102
MXNet C API Calls 57258 544.6150 487.3580 544.6150 28.6285
MXNet C API Concurrency 114515 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 988 0.4730 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4940 2.5730 0.0000 0.0230 0.0005
MXNDArraySetGradState 4940 2.5190 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 988 0.4230 0.0000 0.0010 0.0004
MXAutogradBackwardEx 494 268.5390 0.5250 0.6810 0.5436
MXNDArrayGetContext 7956 3.4270 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16594 421.7230 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1976 4805.6299 0.7010 3.7500 2.4320
FullyConnected 3444 326.3650 0.0080 0.3430 0.0948
argmax 1148 22.4200 0.0050 0.0480 0.0195
_backward_pick 988 15.4770 0.0130 0.0260 0.0157
_backward_mul_scalar 988 4.2630 0.0030 0.0160 0.0043
DeleteVariable 42400 301.0640 0.0010 0.9010 0.0071
Flatten 1148 47.4930 0.0080 0.1120 0.0414
Activation 4592 207.9800 0.0030 0.3430 0.0453
SetValueOp 988 6.3220 0.0050 0.0180 0.0064
pick 988 7.7100 0.0060 0.0280 0.0078
Pooling 2296 3162.0400 0.0640 3.1720 1.3772
Convolution 2296 13021.8467 0.2260 13.2050 5.6715
CopyCPU2CPU 1148 6.6110 0.0030 0.0360 0.0058
log_softmax 988 30.4150 0.0150 0.0500 0.0308
_backward_copy 988 50.8830 0.0220 0.0760 0.0515
_backward_Convolution 1976 18202.1523 2.6370 12.5610 9.2116
_mul_scalar 988 3.4910 0.0020 0.0230 0.0035
_backward_log_softmax 988 16.5040 0.0090 0.0400 0.0167
mean 3124 59.6300 0.0080 0.0730 0.0191
_backward_mean 988 10.7510 0.0080 0.1030 0.0109
_backward_FullyConnected 2964 553.1130 0.0280 0.5940 0.1866
WaitForVar 2136 18.9740 0.0060 0.0280 0.0089
broadcast_equal 1148 5.6850 0.0030 0.0100 0.0050
sgd_update 9880 95.5810 0.0020 0.0640 0.0097
_backward_Activation 3952 289.6680 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39030 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3450 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 494 1963.1030 0.3060 73.5850 3.9739
MXNDArrayCreateEx 575 19.6730 0.0310 0.0640 0.0342
MXNDArrayWaitAll 495 10688.7012 5.6130 24.8060 21.5933
MXNDArraySyncCopyToCPU 1070 8445.9971 0.1180 24.1980 7.8935
MXNDArrayGetDType 1070 0.3750 0.0000 0.0010 0.0004
MXNDArrayGetShape 2715 1.2550 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1150 10.0910 0.0070 0.0280 0.0088
MXNDArrayFree 12825 131.1540 0.0000 0.9140 0.0102
MXNet C API Calls 57365 544.7220 487.3580 544.7220 28.6820
MXNet C API Concurrency 114729 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 990 0.4740 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4950 2.5800 0.0000 0.0230 0.0005
MXNDArraySetGradState 4950 2.5240 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 990 0.4250 0.0000 0.0010 0.0004
MXAutogradBackwardEx 495 269.0720 0.5250 0.6810 0.5436
MXNDArrayGetContext 7970 3.4310 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16625 422.5010 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1980 4815.3511 0.7010 3.7500 2.4320
FullyConnected 3450 326.9190 0.0080 0.3430 0.0948
argmax 1150 22.4590 0.0050 0.0480 0.0195
_backward_pick 990 15.5100 0.0130 0.0260 0.0157
_backward_mul_scalar 990 4.2720 0.0030 0.0160 0.0043
DeleteVariable 42480 301.6260 0.0010 0.9010 0.0071
Flatten 1150 47.5790 0.0080 0.1120 0.0414
Activation 4600 208.3400 0.0030 0.3430 0.0453
SetValueOp 990 6.3350 0.0050 0.0180 0.0064
pick 990 7.7250 0.0060 0.0280 0.0078
Pooling 2300 3167.4099 0.0640 3.1720 1.3771
Convolution 2300 13044.1094 0.2260 13.2050 5.6714
CopyCPU2CPU 1150 6.6210 0.0030 0.0360 0.0058
log_softmax 990 30.4740 0.0150 0.0500 0.0308
_backward_copy 990 50.9960 0.0220 0.0760 0.0515
_backward_Convolution 1980 18238.9551 2.6370 12.5610 9.2116
_mul_scalar 990 3.4970 0.0020 0.0230 0.0035
_backward_log_softmax 990 16.5360 0.0090 0.0400 0.0167
mean 3130 59.7470 0.0080 0.0730 0.0191
_backward_mean 990 10.7720 0.0080 0.1030 0.0109
_backward_FullyConnected 2970 554.2000 0.0280 0.5940 0.1866
WaitForVar 2140 19.0110 0.0060 0.0280 0.0089
broadcast_equal 1150 5.6950 0.0030 0.0100 0.0050
sgd_update 9900 95.7840 0.0020 0.0640 0.0097
_backward_Activation 3960 290.2140 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39104 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3456 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 495 1967.0160 0.3060 73.5850 3.9738
MXNDArrayCreateEx 576 19.7070 0.0310 0.0640 0.0342
MXNDArrayWaitAll 496 10710.4609 5.6130 24.8060 21.5937
MXNDArraySyncCopyToCPU 1072 8460.4434 0.1180 24.1980 7.8922
MXNDArrayGetDType 1072 0.3750 0.0000 0.0010 0.0003
MXNDArrayGetShape 2720 1.2570 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1152 10.1090 0.0070 0.0280 0.0088
MXNDArrayFree 12848 131.3790 0.0000 0.9140 0.0102
MXNet C API Calls 57472 544.8290 487.3580 544.8290 28.7355
MXNet C API Concurrency 114943 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 992 0.4740 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4960 2.5850 0.0000 0.0230 0.0005
MXNDArraySetGradState 4960 2.5280 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 992 0.4250 0.0000 0.0010 0.0004
MXAutogradBackwardEx 496 269.6080 0.5250 0.6810 0.5436
MXNDArrayGetContext 7984 3.4380 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16656 423.2750 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1984 4825.1479 0.7010 3.7500 2.4320
FullyConnected 3456 327.4680 0.0080 0.3430 0.0948
argmax 1152 22.4980 0.0050 0.0480 0.0195
_backward_pick 992 15.5420 0.0130 0.0260 0.0157
_backward_mul_scalar 992 4.2800 0.0030 0.0160 0.0043
DeleteVariable 42560 302.1720 0.0010 0.9010 0.0071
Flatten 1152 47.6630 0.0080 0.1120 0.0414
Activation 4608 208.7010 0.0030 0.3430 0.0453
SetValueOp 992 6.3480 0.0050 0.0180 0.0064
pick 992 7.7410 0.0060 0.0280 0.0078
Pooling 2304 3172.7639 0.0640 3.1720 1.3771
Convolution 2304 13066.2852 0.2260 13.2050 5.6711
CopyCPU2CPU 1152 6.6310 0.0030 0.0360 0.0058
log_softmax 992 30.5330 0.0150 0.0500 0.0308
_backward_copy 992 51.1080 0.0220 0.0760 0.0515
_backward_Convolution 1984 18275.7832 2.6370 12.5610 9.2116
_mul_scalar 992 3.5040 0.0020 0.0230 0.0035
_backward_log_softmax 992 16.5670 0.0090 0.0400 0.0167
mean 3136 59.8640 0.0080 0.0730 0.0191
_backward_mean 992 10.7940 0.0080 0.1030 0.0109
_backward_FullyConnected 2976 555.2870 0.0280 0.5940 0.1866
WaitForVar 2144 19.0480 0.0060 0.0280 0.0089
broadcast_equal 1152 5.7040 0.0030 0.0100 0.0050
sgd_update 9920 96.0210 0.0020 0.0640 0.0097
_backward_Activation 3968 290.8000 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39178 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3462 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 496 1970.9091 0.3060 73.5850 3.9736
MXNDArrayCreateEx 577 19.7410 0.0310 0.0640 0.0342
MXNDArrayWaitAll 497 10732.3311 5.6130 24.8060 21.5942
MXNDArraySyncCopyToCPU 1074 8474.7354 0.1180 24.1980 7.8908
MXNDArrayGetDType 1074 0.3760 0.0000 0.0010 0.0004
MXNDArrayGetShape 2725 1.2620 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1154 10.1260 0.0070 0.0280 0.0088
MXNDArrayFree 12871 131.6030 0.0000 0.9140 0.0102
MXNet C API Calls 57579 544.9360 487.3580 544.9360 28.7890
MXNet C API Concurrency 115157 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 994 0.4750 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4970 2.5890 0.0000 0.0230 0.0005
MXNDArraySetGradState 4970 2.5350 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 994 0.4260 0.0000 0.0010 0.0004
MXAutogradBackwardEx 497 270.1630 0.5250 0.6810 0.5436
MXNDArrayGetContext 7998 3.4480 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16687 424.0760 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1988 4834.8418 0.7010 3.7500 2.4320
FullyConnected 3462 327.9970 0.0080 0.3430 0.0947
argmax 1154 22.5350 0.0050 0.0480 0.0195
_backward_pick 994 15.5750 0.0130 0.0260 0.0157
_backward_mul_scalar 994 4.2890 0.0030 0.0160 0.0043
DeleteVariable 42640 302.7030 0.0010 0.9010 0.0071
Flatten 1154 47.7480 0.0080 0.1120 0.0414
Activation 4616 209.0620 0.0030 0.3430 0.0453
SetValueOp 994 6.3610 0.0050 0.0180 0.0064
pick 994 7.7560 0.0060 0.0280 0.0078
Pooling 2308 3178.1130 0.0640 3.1720 1.3770
Convolution 2308 13088.5283 0.2260 13.2050 5.6709
CopyCPU2CPU 1154 6.6410 0.0030 0.0360 0.0058
log_softmax 994 30.5930 0.0150 0.0500 0.0308
_backward_copy 994 51.2190 0.0220 0.0760 0.0515
_backward_Convolution 1988 18312.8359 2.6370 12.5610 9.2117
_mul_scalar 994 3.5110 0.0020 0.0230 0.0035
_backward_log_softmax 994 16.5990 0.0090 0.0400 0.0167
mean 3142 59.9840 0.0080 0.0730 0.0191
_backward_mean 994 10.8160 0.0080 0.1030 0.0109
_backward_FullyConnected 2982 556.4330 0.0280 0.5940 0.1866
WaitForVar 2148 19.0800 0.0060 0.0280 0.0089
broadcast_equal 1154 5.7130 0.0030 0.0100 0.0050
sgd_update 9940 96.1750 0.0020 0.0640 0.0097
_backward_Activation 3976 291.4170 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39252 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3468 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 497 1974.8101 0.3060 73.5850 3.9735
MXNDArrayCreateEx 578 19.7740 0.0310 0.0640 0.0342
MXNDArrayWaitAll 498 10754.2764 5.6130 24.8060 21.5949
MXNDArraySyncCopyToCPU 1076 8489.7012 0.1180 24.1980 7.8901
MXNDArrayGetDType 1076 0.3760 0.0000 0.0010 0.0003
MXNDArrayGetShape 2730 1.2640 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1156 10.1430 0.0070 0.0280 0.0088
MXNDArrayFree 12894 131.8320 0.0000 0.9140 0.0102
MXNet C API Calls 57686 545.0430 487.3580 545.0430 28.8425
MXNet C API Concurrency 115371 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 996 0.4760 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4980 2.5930 0.0000 0.0230 0.0005
MXNDArraySetGradState 4980 2.5410 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 996 0.4260 0.0000 0.0010 0.0004
MXAutogradBackwardEx 498 270.6970 0.5250 0.6810 0.5436
MXNDArrayGetContext 8012 3.4550 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16718 424.8790 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1992 4844.5439 0.7010 3.7500 2.4320
FullyConnected 3468 328.7980 0.0080 0.3430 0.0948
argmax 1156 22.5800 0.0050 0.0480 0.0195
_backward_pick 996 15.6090 0.0130 0.0260 0.0157
_backward_mul_scalar 996 4.2990 0.0030 0.0160 0.0043
DeleteVariable 42720 303.2580 0.0010 0.9010 0.0071
Flatten 1156 47.8350 0.0080 0.1120 0.0414
Activation 4624 209.4570 0.0030 0.3430 0.0453
SetValueOp 996 6.3740 0.0050 0.0180 0.0064
pick 996 7.7730 0.0060 0.0280 0.0078
Pooling 2312 3183.5200 0.0640 3.1720 1.3770
Convolution 2312 13110.9805 0.2260 13.2050 5.6708
CopyCPU2CPU 1156 6.6520 0.0030 0.0360 0.0058
log_softmax 996 30.6670 0.0150 0.0500 0.0308
_backward_copy 996 51.3290 0.0220 0.0760 0.0515
_backward_Convolution 1992 18350.0625 2.6370 12.5610 9.2119
_mul_scalar 996 3.5180 0.0020 0.0230 0.0035
_backward_log_softmax 996 16.6410 0.0090 0.0400 0.0167
mean 3148 60.0910 0.0080 0.0730 0.0191
_backward_mean 996 10.8380 0.0080 0.1030 0.0109
_backward_FullyConnected 2988 557.8690 0.0280 0.5940 0.1867
WaitForVar 2152 19.1180 0.0060 0.0280 0.0089
broadcast_equal 1156 5.7230 0.0030 0.0100 0.0050
sgd_update 9960 96.3930 0.0020 0.0640 0.0097
_backward_Activation 3984 292.1000 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39326 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3474 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 498 1978.7280 0.3060 73.5850 3.9733
MXNDArrayCreateEx 579 19.8080 0.0310 0.0640 0.0342
MXNDArrayWaitAll 499 10775.9873 5.6130 24.8060 21.5952
MXNDArraySyncCopyToCPU 1078 8504.0449 0.1180 24.1980 7.8887
MXNDArrayGetDType 1078 0.3760 0.0000 0.0010 0.0003
MXNDArrayGetShape 2735 1.2640 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1158 10.1600 0.0070 0.0280 0.0088
MXNDArrayFree 12917 132.0590 0.0000 0.9140 0.0102
MXNet C API Calls 57793 545.1500 487.3580 545.1500 28.8960
MXNet C API Concurrency 115585 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 998 0.4770 0.0000 0.0040 0.0005
MXNDArrayGetGradState 4990 2.5980 0.0000 0.0230 0.0005
MXNDArraySetGradState 4990 2.5460 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 998 0.4270 0.0000 0.0010 0.0004
MXAutogradBackwardEx 499 271.2510 0.5250 0.6810 0.5436
MXNDArrayGetContext 8026 3.4580 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16749 425.6840 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 1996 4854.1631 0.7010 3.7500 2.4319
FullyConnected 3474 329.3520 0.0080 0.3430 0.0948
argmax 1158 22.6190 0.0050 0.0480 0.0195
_backward_pick 998 15.6400 0.0130 0.0260 0.0157
_backward_mul_scalar 998 4.3080 0.0030 0.0160 0.0043
DeleteVariable 42800 303.8020 0.0010 0.9010 0.0071
Flatten 1158 47.9190 0.0080 0.1120 0.0414
Activation 4632 209.8180 0.0030 0.3430 0.0453
SetValueOp 998 6.3870 0.0050 0.0180 0.0064
pick 998 7.7880 0.0060 0.0280 0.0078
Pooling 2316 3188.8840 0.0640 3.1720 1.3769
Convolution 2316 13133.2510 0.2260 13.2050 5.6707
CopyCPU2CPU 1158 6.6620 0.0030 0.0360 0.0058
log_softmax 998 30.7280 0.0150 0.0500 0.0308
_backward_copy 998 51.4420 0.0220 0.0760 0.0515
_backward_Convolution 1996 18386.9238 2.6370 12.5610 9.2119
_mul_scalar 998 3.5250 0.0020 0.0230 0.0035
_backward_log_softmax 998 16.6720 0.0090 0.0400 0.0167
mean 3154 60.2130 0.0080 0.0730 0.0191
_backward_mean 998 10.8600 0.0080 0.1030 0.0109
_backward_FullyConnected 2994 558.9620 0.0280 0.5940 0.1867
WaitForVar 2156 19.1540 0.0060 0.0280 0.0089
broadcast_equal 1158 5.7320 0.0030 0.0100 0.0049
sgd_update 9980 96.6020 0.0020 0.0640 0.0097
_backward_Activation 3992 292.6680 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39400 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3480 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 499 1982.6270 0.3060 73.5850 3.9732
MXNDArrayCreateEx 580 19.8420 0.0310 0.0640 0.0342
MXNDArrayWaitAll 500 10797.6719 5.6130 24.8060 21.5953
MXNDArraySyncCopyToCPU 1080 8518.4492 0.1180 24.1980 7.8875
MXNDArrayGetDType 1080 0.3760 0.0000 0.0010 0.0003
MXNDArrayGetShape 2740 1.2670 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1160 10.1770 0.0070 0.0280 0.0088
MXNDArrayFree 12940 132.2830 0.0000 0.9140 0.0102
MXNet C API Calls 57900 545.2570 487.3580 545.2570 28.9495
MXNet C API Concurrency 115799 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1000 0.4780 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5000 2.6020 0.0000 0.0230 0.0005
MXNDArraySetGradState 5000 2.5510 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1000 0.4270 0.0000 0.0010 0.0004
MXAutogradBackwardEx 500 271.7830 0.5250 0.6810 0.5436
MXNDArrayGetContext 8040 3.4630 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16780 426.4910 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2000 4863.8750 0.7010 3.7500 2.4319
FullyConnected 3480 329.8830 0.0080 0.3430 0.0948
argmax 1160 22.6560 0.0050 0.0480 0.0195
_backward_pick 1000 15.6710 0.0130 0.0260 0.0157
_backward_mul_scalar 1000 4.3160 0.0030 0.0160 0.0043
DeleteVariable 42880 304.3190 0.0010 0.9010 0.0071
Flatten 1160 48.0070 0.0080 0.1120 0.0414
Activation 4640 210.1790 0.0030 0.3430 0.0453
SetValueOp 1000 6.4000 0.0050 0.0180 0.0064
pick 1000 7.8030 0.0060 0.0280 0.0078
Pooling 2320 3194.2820 0.0640 3.1720 1.3768
Convolution 2320 13155.3965 0.2260 13.2050 5.6704
CopyCPU2CPU 1160 6.6720 0.0030 0.0360 0.0058
log_softmax 1000 30.7870 0.0150 0.0500 0.0308
_backward_copy 1000 51.5540 0.0220 0.0760 0.0516
_backward_Convolution 2000 18423.6621 2.6370 12.5610 9.2118
_mul_scalar 1000 3.5320 0.0020 0.0230 0.0035
_backward_log_softmax 1000 16.7030 0.0090 0.0400 0.0167
mean 3160 60.3350 0.0080 0.0730 0.0191
_backward_mean 1000 10.8810 0.0080 0.1030 0.0109
_backward_FullyConnected 3000 560.0500 0.0280 0.5940 0.1867
WaitForVar 2160 19.1900 0.0060 0.0280 0.0089
broadcast_equal 1160 5.7420 0.0030 0.0100 0.0049
sgd_update 10000 96.8060 0.0020 0.0640 0.0097
_backward_Activation 4000 293.2170 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39474 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3486 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 500 1986.5400 0.3060 73.5850 3.9731
MXNDArrayCreateEx 581 19.8750 0.0310 0.0640 0.0342
MXNDArrayWaitAll 501 10819.5723 5.6130 24.8060 21.5960
MXNDArraySyncCopyToCPU 1082 8532.9180 0.1180 24.1980 7.8862
MXNDArrayGetDType 1082 0.3760 0.0000 0.0010 0.0003
MXNDArrayGetShape 2745 1.2690 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1162 10.1950 0.0070 0.0280 0.0088
MXNDArrayFree 12963 132.5110 0.0000 0.9140 0.0102
MXNet C API Calls 58007 545.3640 487.3580 545.3640 29.0030
MXNet C API Concurrency 116013 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1002 0.4790 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5010 2.6080 0.0000 0.0230 0.0005
MXNDArraySetGradState 5010 2.5550 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1002 0.4290 0.0000 0.0010 0.0004
MXAutogradBackwardEx 501 272.3150 0.5250 0.6810 0.5435
MXNDArrayGetContext 8054 3.4700 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16811 427.2660 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2004 4873.5840 0.7010 3.7500 2.4319
FullyConnected 3486 330.4100 0.0080 0.3430 0.0948
argmax 1162 22.6940 0.0050 0.0480 0.0195
_backward_pick 1002 15.7030 0.0130 0.0260 0.0157
_backward_mul_scalar 1002 4.3250 0.0030 0.0160 0.0043
DeleteVariable 42960 304.8390 0.0010 0.9010 0.0071
Flatten 1162 48.0920 0.0080 0.1120 0.0414
Activation 4648 210.5400 0.0030 0.3430 0.0453
SetValueOp 1002 6.4130 0.0050 0.0180 0.0064
pick 1002 7.8180 0.0060 0.0280 0.0078
Pooling 2324 3199.6421 0.0640 3.1720 1.3768
Convolution 2324 13177.6572 0.2260 13.2050 5.6702
CopyCPU2CPU 1162 6.6810 0.0030 0.0360 0.0057
log_softmax 1002 30.8460 0.0150 0.0500 0.0308
_backward_copy 1002 51.6660 0.0220 0.0760 0.0516
_backward_Convolution 2004 18460.7793 2.6370 12.5610 9.2120
_mul_scalar 1002 3.5390 0.0020 0.0230 0.0035
_backward_log_softmax 1002 16.7360 0.0090 0.0400 0.0167
mean 3166 60.4510 0.0080 0.0730 0.0191
_backward_mean 1002 10.9030 0.0080 0.1030 0.0109
_backward_FullyConnected 3006 561.1360 0.0280 0.5940 0.1867
WaitForVar 2164 19.2280 0.0060 0.0280 0.0089
broadcast_equal 1162 5.7520 0.0030 0.0100 0.0050
sgd_update 10020 97.0060 0.0020 0.0640 0.0097
_backward_Activation 4008 293.7860 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39548 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3492 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 501 1990.4290 0.3060 73.5850 3.9729
MXNDArrayCreateEx 582 19.9080 0.0310 0.0640 0.0342
MXNDArrayWaitAll 502 10841.4502 5.6130 24.8060 21.5965
MXNDArraySyncCopyToCPU 1084 8547.8652 0.1180 24.1980 7.8855
MXNDArrayGetDType 1084 0.3770 0.0000 0.0010 0.0003
MXNDArrayGetShape 2750 1.2710 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1164 10.2120 0.0070 0.0280 0.0088
MXNDArrayFree 12986 132.7390 0.0000 0.9140 0.0102
MXNet C API Calls 58114 545.4710 487.3580 545.4710 29.0565
MXNet C API Concurrency 116227 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1004 0.4810 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5020 2.6140 0.0000 0.0230 0.0005
MXNDArraySetGradState 5020 2.5580 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1004 0.4290 0.0000 0.0010 0.0004
MXAutogradBackwardEx 502 272.8670 0.5250 0.6810 0.5436
MXNDArrayGetContext 8068 3.4750 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16842 428.0320 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2008 4883.2822 0.7010 3.7500 2.4319
FullyConnected 3492 331.1930 0.0080 0.3430 0.0948
argmax 1164 22.7390 0.0050 0.0480 0.0195
_backward_pick 1004 15.7380 0.0130 0.0260 0.0157
_backward_mul_scalar 1004 4.3340 0.0030 0.0160 0.0043
DeleteVariable 43040 305.3800 0.0010 0.9010 0.0071
Flatten 1164 48.1770 0.0080 0.1120 0.0414
Activation 4656 210.9380 0.0030 0.3430 0.0453
SetValueOp 1004 6.4270 0.0050 0.0180 0.0064
pick 1004 7.8350 0.0060 0.0280 0.0078
Pooling 2328 3205.0449 0.0640 3.1720 1.3767
Convolution 2328 13200.1035 0.2260 13.2050 5.6701
CopyCPU2CPU 1164 6.6920 0.0030 0.0360 0.0057
log_softmax 1004 30.9180 0.0150 0.0500 0.0308
_backward_copy 1004 51.7750 0.0220 0.0760 0.0516
_backward_Convolution 2008 18497.8750 2.6370 12.5610 9.2121
_mul_scalar 1004 3.5460 0.0020 0.0230 0.0035
_backward_log_softmax 1004 16.7790 0.0090 0.0400 0.0167
mean 3172 60.5580 0.0080 0.0730 0.0191
_backward_mean 1004 10.9260 0.0080 0.1030 0.0109
_backward_FullyConnected 3012 562.5610 0.0280 0.5940 0.1868
WaitForVar 2168 19.2660 0.0060 0.0280 0.0089
broadcast_equal 1164 5.7620 0.0030 0.0100 0.0050
sgd_update 10040 97.2740 0.0020 0.0640 0.0097
_backward_Activation 4016 294.3750 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39622 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3498 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 502 1994.3330 0.3060 73.5850 3.9728
MXNDArrayCreateEx 583 19.9410 0.0310 0.0640 0.0342
MXNDArrayWaitAll 503 10863.1494 5.6130 24.8060 21.5967
MXNDArraySyncCopyToCPU 1086 8562.3711 0.1180 24.1980 7.8843
MXNDArrayGetDType 1086 0.3770 0.0000 0.0010 0.0003
MXNDArrayGetShape 2755 1.2720 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1166 10.2290 0.0070 0.0280 0.0088
MXNDArrayFree 13009 132.9620 0.0000 0.9140 0.0102
MXNet C API Calls 58221 545.5780 487.3580 545.5780 29.1100
MXNet C API Concurrency 116441 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1006 0.4810 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5030 2.6210 0.0000 0.0230 0.0005
MXNDArraySetGradState 5030 2.5640 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1006 0.4290 0.0000 0.0010 0.0004
MXAutogradBackwardEx 503 273.3970 0.5250 0.6810 0.5435
MXNDArrayGetContext 8082 3.4810 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16873 428.8090 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2012 4892.9932 0.7010 3.7500 2.4319
FullyConnected 3498 331.7210 0.0080 0.3430 0.0948
argmax 1166 22.7760 0.0050 0.0480 0.0195
_backward_pick 1006 15.7680 0.0130 0.0260 0.0157
_backward_mul_scalar 1006 4.3430 0.0030 0.0160 0.0043
DeleteVariable 43120 305.9120 0.0010 0.9010 0.0071
Flatten 1166 48.2770 0.0080 0.1120 0.0414
Activation 4664 211.3200 0.0030 0.3430 0.0453
SetValueOp 1006 6.4400 0.0050 0.0180 0.0064
pick 1006 7.8510 0.0060 0.0280 0.0078
Pooling 2332 3210.4021 0.0640 3.1720 1.3767
Convolution 2332 13222.4219 0.2260 13.2050 5.6700
CopyCPU2CPU 1166 6.7020 0.0030 0.0360 0.0057
log_softmax 1006 30.9790 0.0150 0.0500 0.0308
_backward_copy 1006 51.8790 0.0220 0.0760 0.0516
_backward_Convolution 2012 18534.6680 2.6370 12.5610 9.2121
_mul_scalar 1006 3.5530 0.0020 0.0230 0.0035
_backward_log_softmax 1006 16.8110 0.0090 0.0400 0.0167
mean 3178 60.6700 0.0080 0.0730 0.0191
_backward_mean 1006 10.9490 0.0080 0.1030 0.0109
_backward_FullyConnected 3018 563.6320 0.0280 0.5940 0.1868
WaitForVar 2172 19.3020 0.0060 0.0280 0.0089
broadcast_equal 1166 5.7720 0.0030 0.0100 0.0050
sgd_update 10060 97.4730 0.0020 0.0640 0.0097
_backward_Activation 4024 294.9270 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39696 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3504 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 503 1998.2330 0.3060 73.5850 3.9726
MXNDArrayCreateEx 584 19.9750 0.0310 0.0640 0.0342
MXNDArrayWaitAll 504 10884.8516 5.6130 24.8060 21.5969
MXNDArraySyncCopyToCPU 1088 8576.7617 0.1180 24.1980 7.8831
MXNDArrayGetDType 1088 0.3780 0.0000 0.0010 0.0003
MXNDArrayGetShape 2760 1.2740 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1168 10.2460 0.0070 0.0280 0.0088
MXNDArrayFree 13032 133.1780 0.0000 0.9140 0.0102
MXNet C API Calls 58328 545.6850 487.3580 545.6850 29.1635
MXNet C API Concurrency 116655 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1008 0.4820 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5040 2.6270 0.0000 0.0230 0.0005
MXNDArraySetGradState 5040 2.5690 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1008 0.4310 0.0000 0.0010 0.0004
MXAutogradBackwardEx 504 273.9270 0.5250 0.6810 0.5435
MXNDArrayGetContext 8096 3.4890 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16904 429.5820 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2016 4902.6641 0.7010 3.7500 2.4319
FullyConnected 3504 332.2640 0.0080 0.3430 0.0948
argmax 1168 22.8140 0.0050 0.0480 0.0195
_backward_pick 1008 15.8010 0.0130 0.0260 0.0157
_backward_mul_scalar 1008 4.3520 0.0030 0.0160 0.0043
DeleteVariable 43200 306.4380 0.0010 0.9010 0.0071
Flatten 1168 48.3740 0.0080 0.1120 0.0414
Activation 4672 211.6720 0.0030 0.3430 0.0453
SetValueOp 1008 6.4530 0.0050 0.0180 0.0064
pick 1008 7.8660 0.0060 0.0280 0.0078
Pooling 2336 3215.7590 0.0640 3.1720 1.3766
Convolution 2336 13244.6504 0.2260 13.2050 5.6698
CopyCPU2CPU 1168 6.7130 0.0030 0.0360 0.0057
log_softmax 1008 31.0400 0.0150 0.0500 0.0308
_backward_copy 1008 51.9820 0.0220 0.0760 0.0516
_backward_Convolution 2016 18571.4316 2.6370 12.5610 9.2120
_mul_scalar 1008 3.5610 0.0020 0.0230 0.0035
_backward_log_softmax 1008 16.8430 0.0090 0.0400 0.0167
mean 3184 60.7830 0.0080 0.0730 0.0191
_backward_mean 1008 10.9710 0.0080 0.1030 0.0109
_backward_FullyConnected 3024 564.7030 0.0280 0.5940 0.1867
WaitForVar 2176 19.3380 0.0060 0.0280 0.0089
broadcast_equal 1168 5.7810 0.0030 0.0100 0.0049
sgd_update 10080 97.6250 0.0020 0.0640 0.0097
_backward_Activation 4032 295.5140 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39770 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3510 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 504 2002.1270 0.3060 73.5850 3.9725
MXNDArrayCreateEx 585 20.0090 0.0310 0.0640 0.0342
MXNDArrayWaitAll 505 10906.7031 5.6130 24.8060 21.5974
MXNDArraySyncCopyToCPU 1090 8591.1465 0.1180 24.1980 7.8818
MXNDArrayGetDType 1090 0.3780 0.0000 0.0010 0.0003
MXNDArrayGetShape 2765 1.2770 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1170 10.2630 0.0070 0.0280 0.0088
MXNDArrayFree 13055 133.4030 0.0000 0.9140 0.0102
MXNet C API Calls 58435 545.7920 487.3580 545.7920 29.2170
MXNet C API Concurrency 116869 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1010 0.4830 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5050 2.6320 0.0000 0.0230 0.0005
MXNDArraySetGradState 5050 2.5720 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1010 0.4330 0.0000 0.0010 0.0004
MXAutogradBackwardEx 505 274.4600 0.5250 0.6810 0.5435
MXNDArrayGetContext 8110 3.4960 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16935 430.3830 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2020 4912.3579 0.7010 3.7500 2.4319
FullyConnected 3510 332.7910 0.0080 0.3430 0.0948
argmax 1170 22.8510 0.0050 0.0480 0.0195
_backward_pick 1010 15.8320 0.0130 0.0260 0.0157
_backward_mul_scalar 1010 4.3590 0.0030 0.0160 0.0043
DeleteVariable 43280 306.9430 0.0010 0.9010 0.0071
Flatten 1170 48.4510 0.0080 0.1120 0.0414
Activation 4680 212.0240 0.0030 0.3430 0.0453
SetValueOp 1010 6.4670 0.0050 0.0180 0.0064
pick 1010 7.8810 0.0060 0.0280 0.0078
Pooling 2340 3221.1599 0.0640 3.1720 1.3766
Convolution 2340 13266.8164 0.2260 13.2050 5.6696
CopyCPU2CPU 1170 6.7230 0.0030 0.0360 0.0057
log_softmax 1010 31.0990 0.0150 0.0500 0.0308
_backward_copy 1010 52.0850 0.0220 0.0760 0.0516
_backward_Convolution 2020 18608.5469 2.6370 12.5610 9.2122
_mul_scalar 1010 3.5680 0.0020 0.0230 0.0035
_backward_log_softmax 1010 16.8740 0.0090 0.0400 0.0167
mean 3190 60.8990 0.0080 0.0730 0.0191
_backward_mean 1010 10.9920 0.0080 0.1030 0.0109
_backward_FullyConnected 3030 565.7690 0.0280 0.5940 0.1867
WaitForVar 2180 19.3730 0.0060 0.0280 0.0089
broadcast_equal 1170 5.7910 0.0030 0.0100 0.0049
sgd_update 10100 97.8240 0.0020 0.0640 0.0097
_backward_Activation 4040 296.0840 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39844 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3516 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 505 2006.0410 0.3060 73.5850 3.9724
MXNDArrayCreateEx 586 20.0420 0.0310 0.0640 0.0342
MXNDArrayWaitAll 506 10928.5986 5.6130 24.8060 21.5980
MXNDArraySyncCopyToCPU 1092 8605.9844 0.1180 24.1980 7.8809
MXNDArrayGetDType 1092 0.3780 0.0000 0.0010 0.0003
MXNDArrayGetShape 2770 1.2780 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1172 10.2800 0.0070 0.0280 0.0088
MXNDArrayFree 13078 133.6460 0.0000 0.9140 0.0102
MXNet C API Calls 58542 545.8990 487.3580 545.8990 29.2705
MXNet C API Concurrency 117083 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1012 0.4830 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5060 2.6360 0.0000 0.0230 0.0005
MXNDArraySetGradState 5060 2.5780 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1012 0.4330 0.0000 0.0010 0.0004
MXAutogradBackwardEx 506 274.9970 0.5250 0.6810 0.5435
MXNDArrayGetContext 8124 3.4980 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16966 431.1780 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2024 4922.1289 0.7010 3.7500 2.4319
FullyConnected 3516 333.6060 0.0080 0.3430 0.0949
argmax 1172 22.8960 0.0050 0.0480 0.0195
_backward_pick 1012 15.8650 0.0130 0.0260 0.0157
_backward_mul_scalar 1012 4.3660 0.0030 0.0160 0.0043
DeleteVariable 43360 307.4960 0.0010 0.9010 0.0071
Flatten 1172 48.5380 0.0080 0.1120 0.0414
Activation 4688 212.4140 0.0030 0.3430 0.0453
SetValueOp 1012 6.4800 0.0050 0.0180 0.0064
pick 1012 7.8960 0.0060 0.0280 0.0078
Pooling 2344 3226.5181 0.0640 3.1720 1.3765
Convolution 2344 13289.1641 0.2260 13.2050 5.6694
CopyCPU2CPU 1172 6.7330 0.0030 0.0360 0.0057
log_softmax 1012 31.1690 0.0150 0.0500 0.0308
_backward_copy 1012 52.1860 0.0220 0.0760 0.0516
_backward_Convolution 2024 18645.6074 2.6370 12.5610 9.2123
_mul_scalar 1012 3.5740 0.0020 0.0230 0.0035
_backward_log_softmax 1012 16.9170 0.0090 0.0400 0.0167
mean 3196 61.0100 0.0080 0.0730 0.0191
_backward_mean 1012 11.0130 0.0080 0.1030 0.0109
_backward_FullyConnected 3036 567.1840 0.0280 0.5940 0.1868
WaitForVar 2184 19.4070 0.0060 0.0280 0.0089
broadcast_equal 1172 5.8000 0.0030 0.0100 0.0049
sgd_update 10120 98.0950 0.0020 0.0640 0.0097
_backward_Activation 4048 296.6670 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39918 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3522 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 506 2009.9320 0.3060 73.5850 3.9722
MXNDArrayCreateEx 587 20.0750 0.0310 0.0640 0.0342
MXNDArrayWaitAll 507 10950.3340 5.6130 24.8060 21.5983
MXNDArraySyncCopyToCPU 1094 8620.3438 0.1180 24.1980 7.8797
MXNDArrayGetDType 1094 0.3780 0.0000 0.0010 0.0003
MXNDArrayGetShape 2775 1.2780 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1174 10.2980 0.0070 0.0280 0.0088
MXNDArrayFree 13101 133.8690 0.0000 0.9140 0.0102
MXNet C API Calls 58649 546.0060 487.3580 546.0060 29.3240
MXNet C API Concurrency 117297 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1014 0.4830 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5070 2.6390 0.0000 0.0230 0.0005
MXNDArraySetGradState 5070 2.5860 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1014 0.4340 0.0000 0.0010 0.0004
MXAutogradBackwardEx 507 275.5290 0.5250 0.6810 0.5434
MXNDArrayGetContext 8138 3.5050 0.0000 0.0070 0.0004
MXImperativeInvokeEx 16997 431.9570 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2028 4931.8931 0.7010 3.7500 2.4319
FullyConnected 3522 334.1610 0.0080 0.3430 0.0949
argmax 1174 22.9340 0.0050 0.0480 0.0195
_backward_pick 1014 15.8960 0.0130 0.0260 0.0157
_backward_mul_scalar 1014 4.3740 0.0030 0.0160 0.0043
DeleteVariable 43440 308.0290 0.0010 0.9010 0.0071
Flatten 1174 48.6210 0.0080 0.1120 0.0414
Activation 4696 212.7660 0.0030 0.3430 0.0453
SetValueOp 1014 6.4920 0.0050 0.0180 0.0064
pick 1014 7.9110 0.0060 0.0280 0.0078
Pooling 2348 3231.8899 0.0640 3.1720 1.3764
Convolution 2348 13311.3193 0.2260 13.2050 5.6692
CopyCPU2CPU 1174 6.7430 0.0030 0.0360 0.0057
log_softmax 1014 31.2280 0.0150 0.0500 0.0308
_backward_copy 1014 52.2910 0.0220 0.0760 0.0516
_backward_Convolution 2028 18682.4082 2.6370 12.5610 9.2122
_mul_scalar 1014 3.5800 0.0020 0.0230 0.0035
_backward_log_softmax 1014 16.9480 0.0090 0.0400 0.0167
mean 3202 61.1230 0.0080 0.0730 0.0191
_backward_mean 1014 11.0330 0.0080 0.1030 0.0109
_backward_FullyConnected 3042 568.2420 0.0280 0.5940 0.1868
WaitForVar 2188 19.4410 0.0060 0.0280 0.0089
broadcast_equal 1174 5.8090 0.0030 0.0100 0.0049
sgd_update 10140 98.2490 0.0020 0.0640 0.0097
_backward_Activation 4056 297.2790 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 39992 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3528 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 507 2013.8300 0.3060 73.5850 3.9721
MXNDArrayCreateEx 588 20.1130 0.0310 0.0640 0.0342
MXNDArrayWaitAll 508 10972.0645 5.6130 24.8060 21.5986
MXNDArraySyncCopyToCPU 1096 8633.8076 0.1180 24.1980 7.8776
MXNDArrayGetDType 1096 0.3790 0.0000 0.0010 0.0003
MXNDArrayGetShape 2780 1.2790 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1176 10.3170 0.0070 0.0280 0.0088
MXNDArrayFree 13124 134.1020 0.0000 0.9140 0.0102
MXNet C API Calls 58756 546.1130 487.3580 546.1130 29.3775
MXNet C API Concurrency 117511 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1016 0.4840 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5080 2.6450 0.0000 0.0230 0.0005
MXNDArraySetGradState 5080 2.5890 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1016 0.4360 0.0000 0.0010 0.0004
MXAutogradBackwardEx 508 276.0690 0.5250 0.6810 0.5434
MXNDArrayGetContext 8152 3.5150 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17028 432.7870 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2032 4941.5962 0.7010 3.7500 2.4319
FullyConnected 3528 334.7110 0.0080 0.3430 0.0949
argmax 1176 22.9720 0.0050 0.0480 0.0195
_backward_pick 1016 15.9250 0.0130 0.0260 0.0157
_backward_mul_scalar 1016 4.3830 0.0030 0.0160 0.0043
DeleteVariable 43520 308.5650 0.0010 0.9010 0.0071
Flatten 1176 48.6960 0.0080 0.1120 0.0414
Activation 4704 213.1170 0.0030 0.3430 0.0453
SetValueOp 1016 6.5050 0.0050 0.0180 0.0064
pick 1016 7.9260 0.0060 0.0280 0.0078
Pooling 2352 3237.1980 0.0640 3.1720 1.3764
Convolution 2352 13333.4287 0.2260 13.2050 5.6690
CopyCPU2CPU 1176 6.7540 0.0030 0.0360 0.0057
log_softmax 1016 31.2880 0.0150 0.0500 0.0308
_backward_copy 1016 52.3940 0.0220 0.0760 0.0516
_backward_Convolution 2032 18719.1836 2.6370 12.5610 9.2122
_mul_scalar 1016 3.5870 0.0020 0.0230 0.0035
_backward_log_softmax 1016 16.9790 0.0090 0.0400 0.0167
mean 3208 61.2430 0.0080 0.0730 0.0191
_backward_mean 1016 11.0530 0.0080 0.1030 0.0109
_backward_FullyConnected 3048 569.3600 0.0280 0.5940 0.1868
WaitForVar 2192 19.4750 0.0060 0.0280 0.0089
broadcast_equal 1176 5.8180 0.0030 0.0100 0.0049
sgd_update 10160 98.4030 0.0020 0.0640 0.0097
_backward_Activation 4064 297.8700 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40066 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3534 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 508 2015.8350 0.3060 73.5850 3.9682
MXNDArrayCreateEx 589 20.1460 0.0310 0.0640 0.0342
MXNDArrayWaitAll 509 10993.8975 5.6130 24.8060 21.5990
MXNDArraySyncCopyToCPU 1098 8648.1953 0.1180 24.1980 7.8763
MXNDArrayGetDType 1098 0.3800 0.0000 0.0010 0.0003
MXNDArrayGetShape 2785 1.2810 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1178 10.3350 0.0070 0.0280 0.0088
MXNDArrayFree 13147 134.3570 0.0000 0.9140 0.0102
MXNet C API Calls 58863 546.2200 487.3580 546.2200 29.4310
MXNet C API Concurrency 117725 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1018 0.4840 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5090 2.6500 0.0000 0.0230 0.0005
MXNDArraySetGradState 5090 2.5940 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1018 0.4360 0.0000 0.0010 0.0004
MXAutogradBackwardEx 509 276.6050 0.5250 0.6810 0.5434
MXNDArrayGetContext 8166 3.5230 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17059 433.5630 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2036 4951.3071 0.7010 3.7500 2.4319
FullyConnected 3534 335.2310 0.0080 0.3430 0.0949
argmax 1178 23.0090 0.0050 0.0480 0.0195
_backward_pick 1018 15.9550 0.0130 0.0260 0.0157
_backward_mul_scalar 1018 4.3920 0.0030 0.0160 0.0043
DeleteVariable 43600 309.1360 0.0010 0.9010 0.0071
Flatten 1178 48.7730 0.0080 0.1120 0.0414
Activation 4712 213.5070 0.0030 0.3430 0.0453
SetValueOp 1018 6.5160 0.0050 0.0180 0.0064
pick 1018 7.9400 0.0060 0.0280 0.0078
Pooling 2356 3242.5710 0.0640 3.1720 1.3763
Convolution 2356 13355.5957 0.2260 13.2050 5.6688
CopyCPU2CPU 1178 6.7640 0.0030 0.0360 0.0057
log_softmax 1018 31.3470 0.0150 0.0500 0.0308
_backward_copy 1018 52.4980 0.0220 0.0760 0.0516
_backward_Convolution 2036 18756.2207 2.6370 12.5610 9.2123
_mul_scalar 1018 3.5940 0.0020 0.0230 0.0035
_backward_log_softmax 1018 17.0100 0.0090 0.0400 0.0167
mean 3214 61.3600 0.0080 0.0730 0.0191
_backward_mean 1018 11.0730 0.0080 0.1030 0.0109
_backward_FullyConnected 3054 570.4280 0.0280 0.5940 0.1868
WaitForVar 2196 19.5090 0.0060 0.0280 0.0089
broadcast_equal 1178 5.8270 0.0030 0.0100 0.0049
sgd_update 10180 98.6060 0.0020 0.0640 0.0097
_backward_Activation 4072 298.4260 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40140 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3540 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 509 2019.7250 0.3060 73.5850 3.9680
MXNDArrayCreateEx 590 20.1790 0.0310 0.0640 0.0342
MXNDArrayWaitAll 510 11015.7646 5.6130 24.8060 21.5995
MXNDArraySyncCopyToCPU 1100 8662.9971 0.1180 24.1980 7.8755
MXNDArrayGetDType 1100 0.3820 0.0000 0.0010 0.0003
MXNDArrayGetShape 2790 1.2830 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1180 10.3520 0.0070 0.0280 0.0088
MXNDArrayFree 13170 134.5950 0.0000 0.9140 0.0102
MXNet C API Calls 58970 546.3270 487.3580 546.3270 29.4845
MXNet C API Concurrency 117939 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1020 0.4850 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5100 2.6540 0.0000 0.0230 0.0005
MXNDArraySetGradState 5100 2.6000 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1020 0.4380 0.0000 0.0010 0.0004
MXAutogradBackwardEx 510 277.1370 0.5250 0.6810 0.5434
MXNDArrayGetContext 8180 3.5300 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17090 434.3350 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2040 4960.9248 0.7010 3.7500 2.4318
FullyConnected 3540 336.0300 0.0080 0.3430 0.0949
argmax 1180 23.0530 0.0050 0.0480 0.0195
_backward_pick 1020 15.9870 0.0130 0.0260 0.0157
_backward_mul_scalar 1020 4.4000 0.0030 0.0160 0.0043
DeleteVariable 43680 309.6970 0.0010 0.9010 0.0071
Flatten 1180 48.8500 0.0080 0.1120 0.0414
Activation 4720 213.8880 0.0030 0.3430 0.0453
SetValueOp 1020 6.5290 0.0050 0.0180 0.0064
pick 1020 7.9550 0.0060 0.0280 0.0078
Pooling 2360 3247.9451 0.0640 3.1720 1.3762
Convolution 2360 13377.9180 0.2260 13.2050 5.6686
CopyCPU2CPU 1180 6.7740 0.0030 0.0360 0.0057
log_softmax 1020 31.4190 0.0150 0.0500 0.0308
_backward_copy 1020 52.5990 0.0220 0.0760 0.0516
_backward_Convolution 2040 18793.3164 2.6370 12.5610 9.2124
_mul_scalar 1020 3.6000 0.0020 0.0230 0.0035
_backward_log_softmax 1020 17.0510 0.0090 0.0400 0.0167
mean 3220 61.4690 0.0080 0.0730 0.0191
_backward_mean 1020 11.0950 0.0080 0.1030 0.0109
_backward_FullyConnected 3060 571.8550 0.0280 0.5940 0.1869
WaitForVar 2200 19.5440 0.0060 0.0280 0.0089
broadcast_equal 1180 5.8360 0.0030 0.0100 0.0049
sgd_update 10200 98.8160 0.0020 0.0640 0.0097
_backward_Activation 4080 299.1000 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40214 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3546 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 510 2023.6379 0.3060 73.5850 3.9679
MXNDArrayCreateEx 591 20.2120 0.0310 0.0640 0.0342
MXNDArrayWaitAll 511 11037.4736 5.6130 24.8060 21.5998
MXNDArraySyncCopyToCPU 1102 8677.3770 0.1180 24.1980 7.8742
MXNDArrayGetDType 1102 0.3830 0.0000 0.0010 0.0003
MXNDArrayGetShape 2795 1.2830 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1182 10.3690 0.0070 0.0280 0.0088
MXNDArrayFree 13193 134.8430 0.0000 0.9140 0.0102
MXNet C API Calls 59077 546.4340 487.3580 546.4340 29.5380
MXNet C API Concurrency 118153 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1022 0.4860 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5110 2.6610 0.0000 0.0230 0.0005
MXNDArraySetGradState 5110 2.6050 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1022 0.4400 0.0000 0.0010 0.0004
MXAutogradBackwardEx 511 277.6740 0.5250 0.6810 0.5434
MXNDArrayGetContext 8194 3.5360 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17121 435.1120 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2044 4970.5840 0.7010 3.7500 2.4318
FullyConnected 3546 336.5530 0.0080 0.3430 0.0949
argmax 1182 23.0920 0.0050 0.0480 0.0195
_backward_pick 1022 16.0180 0.0130 0.0260 0.0157
_backward_mul_scalar 1022 4.4080 0.0030 0.0160 0.0043
DeleteVariable 43760 310.2930 0.0010 0.9010 0.0071
Flatten 1182 48.9290 0.0080 0.1120 0.0414
Activation 4728 214.2400 0.0030 0.3430 0.0453
SetValueOp 1022 6.5410 0.0050 0.0180 0.0064
pick 1022 7.9700 0.0060 0.0280 0.0078
Pooling 2364 3253.2949 0.0640 3.1720 1.3762
Convolution 2364 13400.0957 0.2260 13.2050 5.6684
CopyCPU2CPU 1182 6.7830 0.0030 0.0360 0.0057
log_softmax 1022 31.4790 0.0150 0.0500 0.0308
_backward_copy 1022 52.7020 0.0220 0.0760 0.0516
_backward_Convolution 2044 18830.1289 2.6370 12.5610 9.2124
_mul_scalar 1022 3.6070 0.0020 0.0230 0.0035
_backward_log_softmax 1022 17.0820 0.0090 0.0400 0.0167
mean 3226 61.5820 0.0080 0.0730 0.0191
_backward_mean 1022 11.1160 0.0080 0.1030 0.0109
_backward_FullyConnected 3066 572.9190 0.0280 0.5940 0.1869
WaitForVar 2204 19.5780 0.0060 0.0280 0.0089
broadcast_equal 1182 5.8450 0.0030 0.0100 0.0049
sgd_update 10220 99.0170 0.0020 0.0640 0.0097
_backward_Activation 4088 299.6800 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40288 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3552 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 511 2027.5410 0.3060 73.5850 3.9678
MXNDArrayCreateEx 592 20.2450 0.0310 0.0640 0.0342
MXNDArrayWaitAll 512 11059.2041 5.6130 24.8060 21.6000
MXNDArraySyncCopyToCPU 1104 8691.6572 0.1180 24.1980 7.8729
MXNDArrayGetDType 1104 0.3840 0.0000 0.0010 0.0003
MXNDArrayGetShape 2800 1.2840 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1184 10.3850 0.0070 0.0280 0.0088
MXNDArrayFree 13216 135.0640 0.0000 0.9140 0.0102
MXNet C API Calls 59184 546.5410 487.3580 546.5410 29.5915
MXNet C API Concurrency 118367 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1024 0.4870 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5120 2.6640 0.0000 0.0230 0.0005
MXNDArraySetGradState 5120 2.6100 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1024 0.4410 0.0000 0.0010 0.0004
MXAutogradBackwardEx 512 278.2100 0.5250 0.6810 0.5434
MXNDArrayGetContext 8208 3.5420 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17152 435.8860 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2048 4980.3008 0.7010 3.7500 2.4318
FullyConnected 3552 337.1450 0.0080 0.3430 0.0949
argmax 1184 23.1300 0.0050 0.0480 0.0195
_backward_pick 1024 16.0480 0.0130 0.0260 0.0157
_backward_mul_scalar 1024 4.4160 0.0030 0.0160 0.0043
DeleteVariable 43840 310.7980 0.0010 0.9010 0.0071
Flatten 1184 49.0060 0.0080 0.1120 0.0414
Activation 4736 214.5930 0.0030 0.3430 0.0453
SetValueOp 1024 6.5530 0.0050 0.0180 0.0064
pick 1024 7.9850 0.0060 0.0280 0.0078
Pooling 2368 3258.6499 0.0640 3.1720 1.3761
Convolution 2368 13422.1904 0.2260 13.2050 5.6682
CopyCPU2CPU 1184 6.7920 0.0030 0.0360 0.0057
log_softmax 1024 31.5400 0.0150 0.0500 0.0308
_backward_copy 1024 52.8050 0.0220 0.0760 0.0516
_backward_Convolution 2048 18866.8633 2.6370 12.5610 9.2123
_mul_scalar 1024 3.6130 0.0020 0.0230 0.0035
_backward_log_softmax 1024 17.1130 0.0090 0.0400 0.0167
mean 3232 61.6900 0.0080 0.0730 0.0191
_backward_mean 1024 11.1370 0.0080 0.1030 0.0109
_backward_FullyConnected 3072 573.9840 0.0280 0.5940 0.1868
WaitForVar 2208 19.6120 0.0060 0.0280 0.0089
broadcast_equal 1184 5.8540 0.0030 0.0100 0.0049
sgd_update 10240 99.1670 0.0020 0.0640 0.0097
_backward_Activation 4096 300.2950 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40362 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3558 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 512 2031.4390 0.3060 73.5850 3.9677
MXNDArrayCreateEx 593 20.2780 0.0310 0.0640 0.0342
MXNDArrayWaitAll 513 11080.9941 5.6130 24.8060 21.6004
MXNDArraySyncCopyToCPU 1106 8706.0225 0.1180 24.1980 7.8716
MXNDArrayGetDType 1106 0.3840 0.0000 0.0010 0.0003
MXNDArrayGetShape 2805 1.2850 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1186 10.4030 0.0070 0.0280 0.0088
MXNDArrayFree 13239 135.2900 0.0000 0.9140 0.0102
MXNet C API Calls 59291 546.6480 487.3580 546.6480 29.6450
MXNet C API Concurrency 118581 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1026 0.4890 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5130 2.6680 0.0000 0.0230 0.0005
MXNDArraySetGradState 5130 2.6150 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1026 0.4410 0.0000 0.0010 0.0004
MXAutogradBackwardEx 513 278.7430 0.5250 0.6810 0.5434
MXNDArrayGetContext 8222 3.5450 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17183 436.6630 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2052 4989.9771 0.7010 3.7500 2.4318
FullyConnected 3558 337.6890 0.0080 0.3430 0.0949
argmax 1186 23.1670 0.0050 0.0480 0.0195
_backward_pick 1026 16.0790 0.0130 0.0260 0.0157
_backward_mul_scalar 1026 4.4240 0.0030 0.0160 0.0043
DeleteVariable 43920 311.3000 0.0010 0.9010 0.0071
Flatten 1186 49.0860 0.0080 0.1120 0.0414
Activation 4744 214.9440 0.0030 0.3430 0.0453
SetValueOp 1026 6.5650 0.0050 0.0180 0.0064
pick 1026 8.0000 0.0060 0.0280 0.0078
Pooling 2372 3264.0239 0.0640 3.1720 1.3761
Convolution 2372 13444.3721 0.2260 13.2050 5.6679
CopyCPU2CPU 1186 6.8020 0.0030 0.0360 0.0057
log_softmax 1026 31.5990 0.0150 0.0500 0.0308
_backward_copy 1026 52.9100 0.0220 0.0760 0.0516
_backward_Convolution 2052 18903.8027 2.6370 12.5610 9.2124
_mul_scalar 1026 3.6200 0.0020 0.0230 0.0035
_backward_log_softmax 1026 17.1440 0.0090 0.0400 0.0167
mean 3238 61.8040 0.0080 0.0730 0.0191
_backward_mean 1026 11.1570 0.0080 0.1030 0.0109
_backward_FullyConnected 3078 575.0490 0.0280 0.5940 0.1868
WaitForVar 2212 19.6460 0.0060 0.0280 0.0089
broadcast_equal 1186 5.8630 0.0030 0.0100 0.0049
sgd_update 10260 99.3160 0.0020 0.0640 0.0097
_backward_Activation 4104 300.9110 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40436 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3564 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 513 2035.3440 0.3060 73.5850 3.9675
MXNDArrayCreateEx 594 20.3110 0.0310 0.0640 0.0342
MXNDArrayWaitAll 514 11102.8213 5.6130 24.8060 21.6008
MXNDArraySyncCopyToCPU 1108 8720.8398 0.1180 24.1980 7.8708
MXNDArrayGetDType 1108 0.3850 0.0000 0.0010 0.0003
MXNDArrayGetShape 2810 1.2880 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1188 10.4200 0.0070 0.0280 0.0088
MXNDArrayFree 13262 135.5200 0.0000 0.9140 0.0102
MXNet C API Calls 59398 546.7550 487.3580 546.7550 29.6985
MXNet C API Concurrency 118795 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1028 0.4890 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5140 2.6750 0.0000 0.0230 0.0005
MXNDArraySetGradState 5140 2.6200 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1028 0.4420 0.0000 0.0010 0.0004
MXAutogradBackwardEx 514 279.2750 0.5250 0.6810 0.5433
MXNDArrayGetContext 8236 3.5490 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17214 437.4290 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2056 4999.6211 0.7010 3.7500 2.4317
FullyConnected 3564 338.4910 0.0080 0.3430 0.0950
argmax 1188 23.2110 0.0050 0.0480 0.0195
_backward_pick 1028 16.1100 0.0130 0.0260 0.0157
_backward_mul_scalar 1028 4.4330 0.0030 0.0160 0.0043
DeleteVariable 44000 311.8330 0.0010 0.9010 0.0071
Flatten 1188 49.1640 0.0080 0.1120 0.0414
Activation 4752 215.3270 0.0030 0.3430 0.0453
SetValueOp 1028 6.5780 0.0050 0.0180 0.0064
pick 1028 8.0160 0.0060 0.0280 0.0078
Pooling 2376 3269.3550 0.0640 3.1720 1.3760
Convolution 2376 13466.7100 0.2260 13.2050 5.6678
CopyCPU2CPU 1188 6.8120 0.0030 0.0360 0.0057
log_softmax 1028 31.6700 0.0150 0.0500 0.0308
_backward_copy 1028 53.0110 0.0220 0.0760 0.0516
_backward_Convolution 2056 18940.8516 2.6370 12.5610 9.2125
_mul_scalar 1028 3.6270 0.0020 0.0230 0.0035
_backward_log_softmax 1028 17.1860 0.0090 0.0400 0.0167
mean 3244 61.9120 0.0080 0.0730 0.0191
_backward_mean 1028 11.1770 0.0080 0.1030 0.0109
_backward_FullyConnected 3084 576.5130 0.0280 0.5940 0.1869
WaitForVar 2216 19.6800 0.0060 0.0280 0.0089
broadcast_equal 1188 5.8720 0.0030 0.0100 0.0049
sgd_update 10280 99.5890 0.0020 0.0640 0.0097
_backward_Activation 4112 301.5000 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40510 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3570 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 514 2039.2500 0.3060 73.5850 3.9674
MXNDArrayCreateEx 595 20.3450 0.0310 0.0640 0.0342
MXNDArrayWaitAll 515 11124.5371 5.6130 24.8060 21.6010
MXNDArraySyncCopyToCPU 1110 8735.2236 0.1180 24.1980 7.8696
MXNDArrayGetDType 1110 0.3860 0.0000 0.0010 0.0003
MXNDArrayGetShape 2815 1.2910 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1190 10.4370 0.0070 0.0280 0.0088
MXNDArrayFree 13285 135.7520 0.0000 0.9140 0.0102
MXNet C API Calls 59505 546.8620 487.3580 546.8620 29.7520
MXNet C API Concurrency 119009 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1030 0.4900 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5150 2.6810 0.0000 0.0230 0.0005
MXNDArraySetGradState 5150 2.6260 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1030 0.4420 0.0000 0.0010 0.0004
MXAutogradBackwardEx 515 279.8350 0.5250 0.6810 0.5434
MXNDArrayGetContext 8250 3.5560 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17245 438.1970 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2060 5009.3550 0.7010 3.7500 2.4317
FullyConnected 3570 339.0190 0.0080 0.3430 0.0950
argmax 1190 23.2490 0.0050 0.0480 0.0195
_backward_pick 1030 16.1400 0.0130 0.0260 0.0157
_backward_mul_scalar 1030 4.4420 0.0030 0.0160 0.0043
DeleteVariable 44080 312.3610 0.0010 0.9010 0.0071
Flatten 1190 49.2430 0.0080 0.1120 0.0414
Activation 4760 215.6790 0.0030 0.3430 0.0453
SetValueOp 1030 6.5910 0.0050 0.0180 0.0064
pick 1030 8.0310 0.0060 0.0280 0.0078
Pooling 2380 3274.7900 0.0640 3.1720 1.3760
Convolution 2380 13488.8838 0.2260 13.2050 5.6676
CopyCPU2CPU 1190 6.8220 0.0030 0.0360 0.0057
log_softmax 1030 31.7300 0.0150 0.0500 0.0308
_backward_copy 1030 53.1150 0.0220 0.0760 0.0516
_backward_Convolution 2060 18977.5938 2.6370 12.5610 9.2124
_mul_scalar 1030 3.6340 0.0020 0.0230 0.0035
_backward_log_softmax 1030 17.2170 0.0090 0.0400 0.0167
mean 3250 62.0260 0.0080 0.0730 0.0191
_backward_mean 1030 11.1970 0.0080 0.1030 0.0109
_backward_FullyConnected 3090 577.5860 0.0280 0.5940 0.1869
WaitForVar 2220 19.7150 0.0060 0.0280 0.0089
broadcast_equal 1190 5.8820 0.0030 0.0100 0.0049
sgd_update 10300 99.7960 0.0020 0.0640 0.0097
_backward_Activation 4120 302.0760 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40584 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3576 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 515 2043.1460 0.3060 73.5850 3.9673
MXNDArrayCreateEx 596 20.3790 0.0310 0.0640 0.0342
MXNDArrayWaitAll 516 11146.2939 5.6130 24.8060 21.6013
MXNDArraySyncCopyToCPU 1112 8749.5498 0.1180 24.1980 7.8683
MXNDArrayGetDType 1112 0.3870 0.0000 0.0010 0.0003
MXNDArrayGetShape 2820 1.2930 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1192 10.4540 0.0070 0.0280 0.0088
MXNDArrayFree 13308 135.9780 0.0000 0.9140 0.0102
MXNet C API Calls 59612 546.9690 487.3580 546.9690 29.8055
MXNet C API Concurrency 119223 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1032 0.4910 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5160 2.6840 0.0000 0.0230 0.0005
MXNDArraySetGradState 5160 2.6330 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1032 0.4420 0.0000 0.0010 0.0004
MXAutogradBackwardEx 516 280.3730 0.5250 0.6810 0.5434
MXNDArrayGetContext 8264 3.5590 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17276 438.9720 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2064 5019.1099 0.7010 3.7500 2.4317
FullyConnected 3576 339.5500 0.0080 0.3430 0.0950
argmax 1192 23.2880 0.0050 0.0480 0.0195
_backward_pick 1032 16.1720 0.0130 0.0260 0.0157
_backward_mul_scalar 1032 4.4510 0.0030 0.0160 0.0043
DeleteVariable 44160 312.8750 0.0010 0.9010 0.0071
Flatten 1192 49.3220 0.0080 0.1120 0.0414
Activation 4768 216.0320 0.0030 0.3430 0.0453
SetValueOp 1032 6.6030 0.0050 0.0180 0.0064
pick 1032 8.0460 0.0060 0.0280 0.0078
Pooling 2384 3280.1389 0.0640 3.1720 1.3759
Convolution 2384 13510.9502 0.2260 13.2050 5.6673
CopyCPU2CPU 1192 6.8310 0.0030 0.0360 0.0057
log_softmax 1032 31.7900 0.0150 0.0500 0.0308
_backward_copy 1032 53.2180 0.0220 0.0760 0.0516
_backward_Convolution 2064 19014.4629 2.6370 12.5610 9.2124
_mul_scalar 1032 3.6410 0.0020 0.0230 0.0035
_backward_log_softmax 1032 17.2480 0.0090 0.0400 0.0167
mean 3256 62.1430 0.0080 0.0730 0.0191
_backward_mean 1032 11.2180 0.0080 0.1030 0.0109
_backward_FullyConnected 3096 578.6530 0.0280 0.5940 0.1869
WaitForVar 2224 19.7490 0.0060 0.0280 0.0089
broadcast_equal 1192 5.8910 0.0030 0.0100 0.0049
sgd_update 10320 99.9980 0.0020 0.0640 0.0097
_backward_Activation 4128 302.6190 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40658 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3582 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 516 2047.0439 0.3060 73.5850 3.9671
MXNDArrayCreateEx 597 20.4120 0.0310 0.0640 0.0342
MXNDArrayWaitAll 517 11168.1562 5.6130 24.8060 21.6018
MXNDArraySyncCopyToCPU 1114 8763.9131 0.1180 24.1980 7.8671
MXNDArrayGetDType 1114 0.3880 0.0000 0.0010 0.0003
MXNDArrayGetShape 2825 1.2960 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1194 10.4720 0.0070 0.0280 0.0088
MXNDArrayFree 13331 136.2010 0.0000 0.9140 0.0102
MXNet C API Calls 59719 547.0760 487.3580 547.0760 29.8590
MXNet C API Concurrency 119437 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1034 0.4920 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5170 2.6910 0.0000 0.0230 0.0005
MXNDArraySetGradState 5170 2.6420 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1034 0.4430 0.0000 0.0010 0.0004
MXAutogradBackwardEx 517 280.9060 0.5250 0.6810 0.5433
MXNDArrayGetContext 8278 3.5630 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17307 439.7480 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2068 5028.8389 0.7010 3.7500 2.4317
FullyConnected 3582 340.0730 0.0080 0.3430 0.0949
argmax 1194 23.3250 0.0050 0.0480 0.0195
_backward_pick 1034 16.2040 0.0130 0.0260 0.0157
_backward_mul_scalar 1034 4.4590 0.0030 0.0160 0.0043
DeleteVariable 44240 313.4040 0.0010 0.9010 0.0071
Flatten 1194 49.4020 0.0080 0.1120 0.0414
Activation 4776 216.3850 0.0030 0.3430 0.0453
SetValueOp 1034 6.6140 0.0050 0.0180 0.0064
pick 1034 8.0610 0.0060 0.0280 0.0078
Pooling 2388 3285.4641 0.0640 3.1720 1.3758
Convolution 2388 13533.1201 0.2260 13.2050 5.6671
CopyCPU2CPU 1194 6.8400 0.0030 0.0360 0.0057
log_softmax 1034 31.8490 0.0150 0.0500 0.0308
_backward_copy 1034 53.3210 0.0220 0.0760 0.0516
_backward_Convolution 2068 19051.5137 2.6370 12.5610 9.2125
_mul_scalar 1034 3.6480 0.0020 0.0230 0.0035
_backward_log_softmax 1034 17.2810 0.0090 0.0400 0.0167
mean 3262 62.2600 0.0080 0.0730 0.0191
_backward_mean 1034 11.2390 0.0080 0.1030 0.0109
_backward_FullyConnected 3102 579.7200 0.0280 0.5940 0.1869
WaitForVar 2228 19.7830 0.0060 0.0280 0.0089
broadcast_equal 1194 5.9000 0.0030 0.0100 0.0049
sgd_update 10340 100.1480 0.0020 0.0640 0.0097
_backward_Activation 4136 303.2450 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40732 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3588 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 517 2050.9541 0.3060 73.5850 3.9670
MXNDArrayCreateEx 598 20.4470 0.0310 0.0640 0.0342
MXNDArrayWaitAll 518 11190.0225 5.6130 24.8060 21.6024
MXNDArraySyncCopyToCPU 1116 8778.6191 0.1180 24.1980 7.8661
MXNDArrayGetDType 1116 0.3880 0.0000 0.0010 0.0003
MXNDArrayGetShape 2830 1.2970 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1196 10.4900 0.0070 0.0280 0.0088
MXNDArrayFree 13354 136.4420 0.0000 0.9140 0.0102
MXNet C API Calls 59826 547.1830 487.3580 547.1830 29.9125
MXNet C API Concurrency 119651 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1036 0.4930 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5180 2.6970 0.0000 0.0230 0.0005
MXNDArraySetGradState 5180 2.6450 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1036 0.4430 0.0000 0.0010 0.0004
MXAutogradBackwardEx 518 281.4390 0.5250 0.6810 0.5433
MXNDArrayGetContext 8292 3.5680 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17338 440.5340 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2072 5038.5391 0.7010 3.7500 2.4317
FullyConnected 3588 340.8460 0.0080 0.3430 0.0950
argmax 1196 23.3680 0.0050 0.0480 0.0195
_backward_pick 1036 16.2350 0.0130 0.0260 0.0157
_backward_mul_scalar 1036 4.4680 0.0030 0.0160 0.0043
DeleteVariable 44320 313.9590 0.0010 0.9010 0.0071
Flatten 1196 49.4790 0.0080 0.1120 0.0414
Activation 4784 216.7680 0.0030 0.3430 0.0453
SetValueOp 1036 6.6250 0.0050 0.0180 0.0064
pick 1036 8.0760 0.0060 0.0280 0.0078
Pooling 2392 3290.8479 0.0640 3.1720 1.3758
Convolution 2392 13555.4395 0.2260 13.2050 5.6670
CopyCPU2CPU 1196 6.8500 0.0030 0.0360 0.0057
log_softmax 1036 31.9210 0.0150 0.0500 0.0308
_backward_copy 1036 53.4220 0.0220 0.0760 0.0516
_backward_Convolution 2072 19088.6270 2.6370 12.5610 9.2127
_mul_scalar 1036 3.6550 0.0020 0.0230 0.0035
_backward_log_softmax 1036 17.3230 0.0090 0.0400 0.0167
mean 3268 62.3660 0.0080 0.0730 0.0191
_backward_mean 1036 11.2600 0.0080 0.1030 0.0109
_backward_FullyConnected 3108 581.1420 0.0280 0.5940 0.1870
WaitForVar 2232 19.8180 0.0060 0.0280 0.0089
broadcast_equal 1196 5.9090 0.0030 0.0100 0.0049
sgd_update 10360 100.3770 0.0020 0.0640 0.0097
_backward_Activation 4144 303.8930 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40806 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3594 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 518 2054.8501 0.3060 73.5850 3.9669
MXNDArrayCreateEx 599 20.4800 0.0310 0.0640 0.0342
MXNDArrayWaitAll 519 11211.8105 5.6130 24.8060 21.6027
MXNDArraySyncCopyToCPU 1118 8793.0068 0.1180 24.1980 7.8649
MXNDArrayGetDType 1118 0.3890 0.0000 0.0010 0.0003
MXNDArrayGetShape 2835 1.2980 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1198 10.5070 0.0070 0.0280 0.0088
MXNDArrayFree 13377 136.6700 0.0000 0.9140 0.0102
MXNet C API Calls 59933 547.2900 487.3580 547.2900 29.9660
MXNet C API Concurrency 119865 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1038 0.4940 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5190 2.7020 0.0000 0.0230 0.0005
MXNDArraySetGradState 5190 2.6500 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1038 0.4450 0.0000 0.0010 0.0004
MXAutogradBackwardEx 519 281.9710 0.5250 0.6810 0.5433
MXNDArrayGetContext 8306 3.5770 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17369 441.3070 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2076 5048.3638 0.7010 3.7500 2.4318
FullyConnected 3594 341.4130 0.0080 0.3430 0.0950
argmax 1198 23.4050 0.0050 0.0480 0.0195
_backward_pick 1038 16.2660 0.0130 0.0260 0.0157
_backward_mul_scalar 1038 4.4760 0.0030 0.0160 0.0043
DeleteVariable 44400 314.4740 0.0010 0.9010 0.0071
Flatten 1198 49.5580 0.0080 0.1120 0.0414
Activation 4792 217.1210 0.0030 0.3430 0.0453
SetValueOp 1038 6.6380 0.0050 0.0180 0.0064
pick 1038 8.0910 0.0060 0.0280 0.0078
Pooling 2396 3296.1899 0.0640 3.1720 1.3757
Convolution 2396 13577.6533 0.2260 13.2050 5.6668
CopyCPU2CPU 1198 6.8610 0.0030 0.0360 0.0057
log_softmax 1038 31.9800 0.0150 0.0500 0.0308
_backward_copy 1038 53.5260 0.0220 0.0760 0.0516
_backward_Convolution 2076 19125.4648 2.6370 12.5610 9.2127
_mul_scalar 1038 3.6620 0.0020 0.0230 0.0035
_backward_log_softmax 1038 17.3560 0.0090 0.0400 0.0167
mean 3274 62.4820 0.0080 0.0730 0.0191
_backward_mean 1038 11.2810 0.0080 0.1030 0.0109
_backward_FullyConnected 3114 582.2110 0.0280 0.5940 0.1870
WaitForVar 2236 19.8560 0.0060 0.0280 0.0089
broadcast_equal 1198 5.9190 0.0030 0.0100 0.0049
sgd_update 10380 100.5810 0.0020 0.0640 0.0097
_backward_Activation 4152 304.4430 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40880 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3600 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 519 2058.7881 0.3060 73.5850 3.9668
MXNDArrayCreateEx 600 20.5170 0.0310 0.0640 0.0342
MXNDArrayWaitAll 520 11233.5146 5.6130 24.8060 21.6029
MXNDArraySyncCopyToCPU 1120 8807.4004 0.1180 24.1980 7.8637
MXNDArrayGetDType 1120 0.3900 0.0000 0.0010 0.0003
MXNDArrayGetShape 2840 1.3010 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1200 10.5250 0.0070 0.0280 0.0088
MXNDArrayFree 13400 136.9060 0.0000 0.9140 0.0102
MXNet C API Calls 60040 547.3970 487.3580 547.3970 30.0195
MXNet C API Concurrency 120079 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1040 0.4940 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5200 2.7070 0.0000 0.0230 0.0005
MXNDArraySetGradState 5200 2.6560 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1040 0.4450 0.0000 0.0010 0.0004
MXAutogradBackwardEx 520 282.5190 0.5250 0.6810 0.5433
MXNDArrayGetContext 8320 3.5810 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17400 442.1300 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2080 5058.1201 0.7010 3.7500 2.4318
FullyConnected 3600 341.9600 0.0080 0.3430 0.0950
argmax 1200 23.4440 0.0050 0.0480 0.0195
_backward_pick 1040 16.2990 0.0130 0.0260 0.0157
_backward_mul_scalar 1040 4.4840 0.0030 0.0160 0.0043
DeleteVariable 44480 315.0500 0.0010 0.9010 0.0071
Flatten 1200 49.6610 0.0080 0.1120 0.0414
Activation 4800 217.4930 0.0030 0.3430 0.0453
SetValueOp 1040 6.6500 0.0050 0.0180 0.0064
pick 1040 8.1060 0.0060 0.0280 0.0078
Pooling 2400 3301.5620 0.0640 3.1720 1.3757
Convolution 2400 13599.8574 0.2260 13.2050 5.6666
CopyCPU2CPU 1200 6.8710 0.0030 0.0360 0.0057
log_softmax 1040 32.0420 0.0150 0.0500 0.0308
_backward_copy 1040 53.6290 0.0220 0.0760 0.0516
_backward_Convolution 2080 19162.1816 2.6370 12.5610 9.2126
_mul_scalar 1040 3.6690 0.0020 0.0230 0.0035
_backward_log_softmax 1040 17.3890 0.0090 0.0400 0.0167
mean 3280 62.6000 0.0080 0.0730 0.0191
_backward_mean 1040 11.3040 0.0080 0.1030 0.0109
_backward_FullyConnected 3120 583.3000 0.0280 0.5940 0.1870
WaitForVar 2240 19.8910 0.0060 0.0280 0.0089
broadcast_equal 1200 5.9290 0.0030 0.0100 0.0049
sgd_update 10400 100.7870 0.0020 0.0640 0.0097
_backward_Activation 4160 305.0350 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 40954 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3606 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 520 2062.7930 0.3060 73.5850 3.9669
MXNDArrayCreateEx 601 20.5540 0.0310 0.0640 0.0342
MXNDArrayWaitAll 521 11255.4268 5.6130 24.8060 21.6035
MXNDArraySyncCopyToCPU 1122 8821.7666 0.1180 24.1980 7.8625
MXNDArrayGetDType 1122 0.3900 0.0000 0.0010 0.0003
MXNDArrayGetShape 2845 1.3030 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1202 10.5440 0.0070 0.0280 0.0088
MXNDArrayFree 13423 137.1620 0.0000 0.9140 0.0102
MXNet C API Calls 60147 547.5040 487.3580 547.5040 30.0730
MXNet C API Concurrency 120293 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1042 0.4950 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5210 2.7140 0.0000 0.0230 0.0005
MXNDArraySetGradState 5210 2.6600 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1042 0.4450 0.0000 0.0010 0.0004
MXAutogradBackwardEx 521 283.0820 0.5250 0.6810 0.5433
MXNDArrayGetContext 8334 3.5870 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17431 442.9460 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2084 5067.9341 0.7010 3.7500 2.4318
FullyConnected 3606 342.5020 0.0080 0.3430 0.0950
argmax 1202 23.4820 0.0050 0.0480 0.0195
_backward_pick 1042 16.3320 0.0130 0.0260 0.0157
_backward_mul_scalar 1042 4.4930 0.0030 0.0160 0.0043
DeleteVariable 44560 315.6310 0.0010 0.9010 0.0071
Flatten 1202 49.7730 0.0080 0.1120 0.0414
Activation 4808 217.8610 0.0030 0.3430 0.0453
SetValueOp 1042 6.6630 0.0050 0.0180 0.0064
pick 1042 8.1220 0.0060 0.0280 0.0078
Pooling 2404 3306.8811 0.0640 3.1720 1.3756
Convolution 2404 13622.1494 0.2260 13.2050 5.6665
CopyCPU2CPU 1202 6.8820 0.0030 0.0360 0.0057
log_softmax 1042 32.1520 0.0150 0.0550 0.0309
_backward_copy 1042 53.7320 0.0220 0.0760 0.0516
_backward_Convolution 2084 19199.2598 2.6370 12.5610 9.2127
_mul_scalar 1042 3.6760 0.0020 0.0230 0.0035
_backward_log_softmax 1042 17.4220 0.0090 0.0400 0.0167
mean 3286 62.7190 0.0080 0.0730 0.0191
_backward_mean 1042 11.3270 0.0080 0.1030 0.0109
_backward_FullyConnected 3126 584.3920 0.0280 0.5940 0.1869
WaitForVar 2244 19.9250 0.0060 0.0280 0.0089
broadcast_equal 1202 5.9410 0.0030 0.0100 0.0049
sgd_update 10420 100.9890 0.0020 0.0640 0.0097
_backward_Activation 4168 305.5980 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41028 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3612 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 521 2066.7920 0.3060 73.5850 3.9670
MXNDArrayCreateEx 602 20.5910 0.0310 0.0640 0.0342
MXNDArrayWaitAll 522 11277.2764 5.6130 24.8060 21.6040
MXNDArraySyncCopyToCPU 1124 8836.6328 0.1180 24.1980 7.8618
MXNDArrayGetDType 1124 0.3910 0.0000 0.0010 0.0003
MXNDArrayGetShape 2850 1.3050 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1204 10.5630 0.0070 0.0280 0.0088
MXNDArrayFree 13446 137.4320 0.0000 0.9140 0.0102
MXNet C API Calls 60254 547.6110 487.3580 547.6110 30.1265
MXNet C API Concurrency 120507 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1044 0.4960 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5220 2.7200 0.0000 0.0230 0.0005
MXNDArraySetGradState 5220 2.6650 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1044 0.4450 0.0000 0.0010 0.0004
MXAutogradBackwardEx 522 283.6490 0.5250 0.6810 0.5434
MXNDArrayGetContext 8348 3.5930 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17462 443.7620 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2088 5077.6631 0.7010 3.7500 2.4318
FullyConnected 3612 343.3060 0.0080 0.3430 0.0950
argmax 1204 23.5280 0.0050 0.0480 0.0195
_backward_pick 1044 16.3670 0.0130 0.0260 0.0157
_backward_mul_scalar 1044 4.5020 0.0030 0.0160 0.0043
DeleteVariable 44640 316.2510 0.0010 0.9010 0.0071
Flatten 1204 49.8850 0.0080 0.1120 0.0414
Activation 4816 218.2640 0.0030 0.3430 0.0453
SetValueOp 1044 6.6750 0.0050 0.0180 0.0064
pick 1044 8.1380 0.0060 0.0280 0.0078
Pooling 2408 3312.2739 0.0640 3.1720 1.3755
Convolution 2408 13644.6328 0.2260 13.2050 5.6664
CopyCPU2CPU 1204 6.8920 0.0030 0.0360 0.0057
log_softmax 1044 32.2270 0.0150 0.0550 0.0309
_backward_copy 1044 53.8320 0.0220 0.0760 0.0516
_backward_Convolution 2088 19236.3105 2.6370 12.5610 9.2128
_mul_scalar 1044 3.6830 0.0020 0.0230 0.0035
_backward_log_softmax 1044 17.4660 0.0090 0.0400 0.0167
mean 3292 62.8270 0.0080 0.0730 0.0191
_backward_mean 1044 11.3500 0.0080 0.1030 0.0109
_backward_FullyConnected 3132 585.8360 0.0280 0.5940 0.1870
WaitForVar 2248 19.9590 0.0060 0.0280 0.0089
broadcast_equal 1204 5.9630 0.0030 0.0110 0.0050
sgd_update 10440 101.2600 0.0020 0.0640 0.0097
_backward_Activation 4176 306.1670 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41102 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3618 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 522 2070.7151 0.3060 73.5850 3.9669
MXNDArrayCreateEx 603 20.6240 0.0310 0.0640 0.0342
MXNDArrayWaitAll 523 11299.0039 5.6130 24.8060 21.6042
MXNDArraySyncCopyToCPU 1126 8850.8135 0.1180 24.1980 7.8604
MXNDArrayGetDType 1126 0.3930 0.0000 0.0010 0.0003
MXNDArrayGetShape 2855 1.3080 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1206 10.5810 0.0070 0.0280 0.0088
MXNDArrayFree 13469 137.6650 0.0000 0.9140 0.0102
MXNet C API Calls 60361 547.7180 487.3580 547.7180 30.1800
MXNet C API Concurrency 120721 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1046 0.4970 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5230 2.7260 0.0000 0.0230 0.0005
MXNDArraySetGradState 5230 2.6720 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1046 0.4450 0.0000 0.0010 0.0004
MXAutogradBackwardEx 523 284.2140 0.5250 0.6810 0.5434
MXNDArrayGetContext 8362 3.6000 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17493 444.5720 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2092 5087.3569 0.7010 3.7500 2.4318
FullyConnected 3618 343.8570 0.0080 0.3430 0.0950
argmax 1206 23.5670 0.0050 0.0480 0.0195
_backward_pick 1046 16.3980 0.0130 0.0260 0.0157
_backward_mul_scalar 1046 4.5090 0.0030 0.0160 0.0043
DeleteVariable 44720 316.7920 0.0010 0.9010 0.0071
Flatten 1206 49.9990 0.0080 0.1120 0.0415
Activation 4824 218.6210 0.0030 0.3430 0.0453
SetValueOp 1046 6.6880 0.0050 0.0180 0.0064
pick 1046 8.1530 0.0060 0.0280 0.0078
Pooling 2412 3317.5891 0.0640 3.1720 1.3755
Convolution 2412 13666.8594 0.2260 13.2050 5.6662
CopyCPU2CPU 1206 6.9020 0.0030 0.0360 0.0057
log_softmax 1046 32.2880 0.0150 0.0550 0.0309
_backward_copy 1046 53.9350 0.0220 0.0760 0.0516
_backward_Convolution 2092 19273.0996 2.6370 12.5610 9.2128
_mul_scalar 1046 3.6900 0.0020 0.0230 0.0035
_backward_log_softmax 1046 17.4970 0.0090 0.0400 0.0167
mean 3298 62.9400 0.0080 0.0730 0.0191
_backward_mean 1046 11.3720 0.0080 0.1030 0.0109
_backward_FullyConnected 3138 586.9000 0.0280 0.5940 0.1870
WaitForVar 2252 19.9940 0.0060 0.0280 0.0089
broadcast_equal 1206 5.9720 0.0030 0.0110 0.0050
sgd_update 10460 101.4620 0.0020 0.0640 0.0097
_backward_Activation 4184 306.7190 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41176 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3624 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 523 2074.6050 0.3060 73.5850 3.9667
MXNDArrayCreateEx 604 20.6590 0.0310 0.0640 0.0342
MXNDArrayWaitAll 524 11320.7051 5.6130 24.8060 21.6044
MXNDArraySyncCopyToCPU 1128 8865.1377 0.1180 24.1980 7.8592
MXNDArrayGetDType 1128 0.3930 0.0000 0.0010 0.0003
MXNDArrayGetShape 2860 1.3100 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1208 10.5970 0.0070 0.0280 0.0088
MXNDArrayFree 13492 137.8940 0.0000 0.9140 0.0102
MXNet C API Calls 60468 547.8250 487.3580 547.8250 30.2335
MXNet C API Concurrency 120935 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1048 0.4980 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5240 2.7310 0.0000 0.0230 0.0005
MXNDArraySetGradState 5240 2.6760 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1048 0.4460 0.0000 0.0010 0.0004
MXAutogradBackwardEx 524 284.7550 0.5250 0.6810 0.5434
MXNDArrayGetContext 8376 3.6030 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17524 445.3750 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2096 5097.0718 0.7010 3.7500 2.4318
FullyConnected 3624 344.3800 0.0080 0.3430 0.0950
argmax 1208 23.6050 0.0050 0.0480 0.0195
_backward_pick 1048 16.4290 0.0130 0.0260 0.0157
_backward_mul_scalar 1048 4.5180 0.0030 0.0160 0.0043
DeleteVariable 44800 317.3040 0.0010 0.9010 0.0071
Flatten 1208 50.0800 0.0080 0.1120 0.0415
Activation 4832 218.9720 0.0030 0.3430 0.0453
SetValueOp 1048 6.7000 0.0050 0.0180 0.0064
pick 1048 8.1680 0.0060 0.0280 0.0078
Pooling 2416 3322.9851 0.0640 3.1720 1.3754
Convolution 2416 13688.9688 0.2260 13.2050 5.6660
CopyCPU2CPU 1208 6.9110 0.0030 0.0360 0.0057
log_softmax 1048 32.3470 0.0150 0.0550 0.0309
_backward_copy 1048 54.0380 0.0220 0.0760 0.0516
_backward_Convolution 2096 19309.8496 2.6370 12.5610 9.2127
_mul_scalar 1048 3.6970 0.0020 0.0230 0.0035
_backward_log_softmax 1048 17.5290 0.0090 0.0400 0.0167
mean 3304 63.0530 0.0080 0.0730 0.0191
_backward_mean 1048 11.3930 0.0080 0.1030 0.0109
_backward_FullyConnected 3144 587.9690 0.0280 0.5940 0.1870
WaitForVar 2256 20.0270 0.0060 0.0280 0.0089
broadcast_equal 1208 5.9820 0.0030 0.0110 0.0050
sgd_update 10480 101.6130 0.0020 0.0640 0.0097
_backward_Activation 4192 307.3520 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41250 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3630 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 524 2078.5090 0.3060 73.5850 3.9666
MXNDArrayCreateEx 605 20.6920 0.0310 0.0640 0.0342
MXNDArrayWaitAll 525 11332.4287 5.6130 24.8060 21.5856
MXNDArraySyncCopyToCPU 1130 8889.0127 0.1180 24.1980 7.8664
MXNDArrayGetDType 1130 0.3950 0.0000 0.0010 0.0003
MXNDArrayGetShape 2865 1.3120 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1210 10.6160 0.0070 0.0280 0.0088
MXNDArrayFree 13515 138.1260 0.0000 0.9140 0.0102
MXNet C API Calls 60575 547.9320 487.3580 547.9320 30.2870
MXNet C API Concurrency 121149 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1050 0.4980 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5250 2.7340 0.0000 0.0230 0.0005
MXNDArraySetGradState 5250 2.6810 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1050 0.4470 0.0000 0.0010 0.0004
MXAutogradBackwardEx 525 285.2880 0.5250 0.6810 0.5434
MXNDArrayGetContext 8390 3.6100 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17555 446.1520 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2100 5106.8018 0.7010 3.7500 2.4318
FullyConnected 3630 344.9030 0.0080 0.3430 0.0950
argmax 1210 23.6440 0.0050 0.0480 0.0195
_backward_pick 1050 16.4600 0.0130 0.0260 0.0157
_backward_mul_scalar 1050 4.5260 0.0030 0.0160 0.0043
DeleteVariable 44880 317.8420 0.0010 0.9010 0.0071
Flatten 1210 50.1590 0.0080 0.1120 0.0415
Activation 4840 219.3240 0.0030 0.3430 0.0453
SetValueOp 1050 6.7130 0.0050 0.0180 0.0064
pick 1050 8.1820 0.0060 0.0280 0.0078
Pooling 2420 3328.3320 0.0640 3.1720 1.3753
Convolution 2420 13711.1406 0.2260 13.2050 5.6658
CopyCPU2CPU 1210 6.9720 0.0030 0.0360 0.0058
log_softmax 1050 32.4080 0.0150 0.0550 0.0309
_backward_copy 1050 54.1420 0.0220 0.0760 0.0516
_backward_Convolution 2100 19346.8320 2.6370 12.5610 9.2128
_mul_scalar 1050 3.7040 0.0020 0.0230 0.0035
_backward_log_softmax 1050 17.5600 0.0090 0.0400 0.0167
mean 3310 63.1280 0.0080 0.0730 0.0191
_backward_mean 1050 11.4140 0.0080 0.1030 0.0109
_backward_FullyConnected 3150 589.0370 0.0280 0.5940 0.1870
WaitForVar 2260 20.0580 0.0060 0.0280 0.0089
broadcast_equal 1210 5.9910 0.0030 0.0110 0.0050
sgd_update 10500 101.8110 0.0020 0.0640 0.0097
_backward_Activation 4200 307.9320 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41324 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3636 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 525 2082.4319 0.3060 73.5850 3.9665
MXNDArrayCreateEx 606 20.7270 0.0310 0.0640 0.0342
MXNDArrayWaitAll 526 11354.2930 5.6130 24.8060 21.5861
MXNDArraySyncCopyToCPU 1132 8903.8838 0.1180 24.1980 7.8656
MXNDArrayGetDType 1132 0.3960 0.0000 0.0010 0.0003
MXNDArrayGetShape 2870 1.3140 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1212 10.6320 0.0070 0.0280 0.0088
MXNDArrayFree 13538 138.3640 0.0000 0.9140 0.0102
MXNet C API Calls 60682 548.0390 487.3580 548.0390 30.3405
MXNet C API Concurrency 121363 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1052 0.4990 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5260 2.7400 0.0000 0.0230 0.0005
MXNDArraySetGradState 5260 2.6860 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1052 0.4470 0.0000 0.0010 0.0004
MXAutogradBackwardEx 526 285.8580 0.5250 0.6810 0.5435
MXNDArrayGetContext 8404 3.6170 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17586 446.9450 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2104 5116.6118 0.7010 3.7500 2.4318
FullyConnected 3636 345.7190 0.0080 0.3430 0.0951
argmax 1212 23.6880 0.0050 0.0480 0.0195
_backward_pick 1052 16.4930 0.0130 0.0260 0.0157
_backward_mul_scalar 1052 4.5350 0.0030 0.0160 0.0043
DeleteVariable 44960 318.4010 0.0010 0.9010 0.0071
Flatten 1212 50.2510 0.0080 0.1120 0.0415
Activation 4848 219.7160 0.0030 0.3430 0.0453
SetValueOp 1052 6.7260 0.0050 0.0180 0.0064
pick 1052 8.1980 0.0060 0.0280 0.0078
Pooling 2424 3333.7490 0.0640 3.1720 1.3753
Convolution 2424 13733.5361 0.2260 13.2050 5.6657
CopyCPU2CPU 1212 6.9830 0.0030 0.0360 0.0058
log_softmax 1052 32.4810 0.0150 0.0550 0.0309
_backward_copy 1052 54.2430 0.0220 0.0760 0.0516
_backward_Convolution 2104 19383.8203 2.6370 12.5610 9.2128
_mul_scalar 1052 3.7110 0.0020 0.0230 0.0035
_backward_log_softmax 1052 17.6030 0.0090 0.0400 0.0167
mean 3316 63.2350 0.0080 0.0730 0.0191
_backward_mean 1052 11.4350 0.0080 0.1030 0.0109
_backward_FullyConnected 3156 590.4760 0.0280 0.5940 0.1871
WaitForVar 2264 20.0930 0.0060 0.0280 0.0089
broadcast_equal 1212 6.0010 0.0030 0.0110 0.0050
sgd_update 10520 102.0830 0.0020 0.0640 0.0097
_backward_Activation 4208 308.5380 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41398 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3642 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 526 2086.3850 0.3060 73.5850 3.9665
MXNDArrayCreateEx 607 20.7610 0.0310 0.0640 0.0342
MXNDArrayWaitAll 527 11375.9863 5.6130 24.8060 21.5863
MXNDArraySyncCopyToCPU 1134 8918.2988 0.1180 24.1980 7.8645
MXNDArrayGetDType 1134 0.3980 0.0000 0.0010 0.0004
MXNDArrayGetShape 2875 1.3160 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1214 10.6500 0.0070 0.0280 0.0088
MXNDArrayFree 13561 138.6060 0.0000 0.9140 0.0102
MXNet C API Calls 60789 548.1460 487.3580 548.1460 30.3940
MXNet C API Concurrency 121577 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1054 0.5000 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5270 2.7430 0.0000 0.0230 0.0005
MXNDArraySetGradState 5270 2.6880 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1054 0.4480 0.0000 0.0010 0.0004
MXAutogradBackwardEx 527 286.4020 0.5250 0.6810 0.5435
MXNDArrayGetContext 8418 3.6250 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17617 447.7250 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2108 5126.3081 0.7010 3.7500 2.4318
FullyConnected 3642 346.2770 0.0080 0.3430 0.0951
argmax 1214 23.7740 0.0050 0.0480 0.0196
_backward_pick 1054 16.5260 0.0130 0.0260 0.0157
_backward_mul_scalar 1054 4.5430 0.0030 0.0160 0.0043
DeleteVariable 45040 318.9560 0.0010 0.9010 0.0071
Flatten 1214 50.3390 0.0080 0.1120 0.0415
Activation 4856 220.0740 0.0030 0.3430 0.0453
SetValueOp 1054 6.7390 0.0050 0.0180 0.0064
pick 1054 8.2140 0.0060 0.0280 0.0078
Pooling 2428 3339.0879 0.0640 3.1720 1.3752
Convolution 2428 13755.7637 0.2260 13.2050 5.6655
CopyCPU2CPU 1214 6.9930 0.0030 0.0360 0.0058
log_softmax 1054 32.5410 0.0150 0.0550 0.0309
_backward_copy 1054 54.3450 0.0220 0.0760 0.0516
_backward_Convolution 2108 19420.5547 2.6370 12.5610 9.2128
_mul_scalar 1054 3.7180 0.0020 0.0230 0.0035
_backward_log_softmax 1054 17.6360 0.0090 0.0400 0.0167
mean 3322 63.3500 0.0080 0.0730 0.0191
_backward_mean 1054 11.4560 0.0080 0.1030 0.0109
_backward_FullyConnected 3162 591.5530 0.0280 0.5940 0.1871
WaitForVar 2268 20.1280 0.0060 0.0280 0.0089
broadcast_equal 1214 6.0120 0.0030 0.0110 0.0050
sgd_update 10540 102.2350 0.0020 0.0640 0.0097
_backward_Activation 4216 309.1680 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41472 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3648 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 527 2090.3120 0.3060 73.5850 3.9664
MXNDArrayCreateEx 608 20.7940 0.0310 0.0640 0.0342
MXNDArrayWaitAll 528 11397.6670 5.6130 24.8060 21.5865
MXNDArraySyncCopyToCPU 1136 8932.5859 0.1180 24.1980 7.8632
MXNDArrayGetDType 1136 0.3990 0.0000 0.0010 0.0004
MXNDArrayGetShape 2880 1.3180 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1216 10.6670 0.0070 0.0280 0.0088
MXNDArrayFree 13584 138.8420 0.0000 0.9140 0.0102
MXNet C API Calls 60896 548.2530 487.3580 548.2530 30.4475
MXNet C API Concurrency 121791 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1056 0.5010 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5280 2.7490 0.0000 0.0230 0.0005
MXNDArraySetGradState 5280 2.6920 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1056 0.4490 0.0000 0.0010 0.0004
MXAutogradBackwardEx 528 286.9810 0.5250 0.6810 0.5435
MXNDArrayGetContext 8432 3.6330 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17648 448.5170 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2112 5135.8911 0.7010 3.7500 2.4318
FullyConnected 3648 346.8070 0.0080 0.3430 0.0951
argmax 1216 23.8140 0.0050 0.0480 0.0196
_backward_pick 1056 16.5580 0.0130 0.0260 0.0157
_backward_mul_scalar 1056 4.5510 0.0030 0.0160 0.0043
DeleteVariable 45120 319.5120 0.0010 0.9010 0.0071
Flatten 1216 50.4250 0.0080 0.1120 0.0415
Activation 4864 220.4320 0.0030 0.3430 0.0453
SetValueOp 1056 6.7520 0.0050 0.0180 0.0064
pick 1056 8.2300 0.0060 0.0280 0.0078
Pooling 2432 3344.4900 0.0640 3.1720 1.3752
Convolution 2432 13777.9043 0.2260 13.2050 5.6653
CopyCPU2CPU 1216 7.0030 0.0030 0.0360 0.0058
log_softmax 1056 32.6010 0.0150 0.0550 0.0309
_backward_copy 1056 54.4500 0.0220 0.0760 0.0516
_backward_Convolution 2112 19457.3105 2.6370 12.5610 9.2127
_mul_scalar 1056 3.7250 0.0020 0.0230 0.0035
_backward_log_softmax 1056 17.6690 0.0090 0.0400 0.0167
mean 3328 63.4580 0.0080 0.0730 0.0191
_backward_mean 1056 11.4780 0.0080 0.1030 0.0109
_backward_FullyConnected 3168 592.6280 0.0280 0.5940 0.1871
WaitForVar 2272 20.1620 0.0060 0.0280 0.0089
broadcast_equal 1216 6.0230 0.0030 0.0110 0.0050
sgd_update 10560 102.3850 0.0020 0.0640 0.0097
_backward_Activation 4224 309.7860 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41546 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3654 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 528 2094.2200 0.3060 73.5850 3.9663
MXNDArrayCreateEx 609 20.8270 0.0310 0.0640 0.0342
MXNDArrayWaitAll 529 11419.4873 5.6130 24.8060 21.5869
MXNDArraySyncCopyToCPU 1138 8946.9902 0.1180 24.1980 7.8620
MXNDArrayGetDType 1138 0.4000 0.0000 0.0010 0.0004
MXNDArrayGetShape 2885 1.3200 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1218 10.6840 0.0070 0.0280 0.0088
MXNDArrayFree 13607 139.0710 0.0000 0.9140 0.0102
MXNet C API Calls 61003 548.3600 487.3580 548.3600 30.5010
MXNet C API Concurrency 122005 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1058 0.5020 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5290 2.7550 0.0000 0.0230 0.0005
MXNDArraySetGradState 5290 2.6930 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1058 0.4500 0.0000 0.0010 0.0004
MXAutogradBackwardEx 529 287.5150 0.5250 0.6810 0.5435
MXNDArrayGetContext 8446 3.6410 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17679 449.3000 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2116 5145.6421 0.7010 3.7500 2.4318
FullyConnected 3654 347.3490 0.0080 0.3430 0.0951
argmax 1218 23.8520 0.0050 0.0480 0.0196
_backward_pick 1058 16.5900 0.0130 0.0260 0.0157
_backward_mul_scalar 1058 4.5600 0.0030 0.0160 0.0043
DeleteVariable 45200 320.0580 0.0010 0.9010 0.0071
Flatten 1218 50.5030 0.0080 0.1120 0.0415
Activation 4872 220.7860 0.0030 0.3430 0.0453
SetValueOp 1058 6.7650 0.0050 0.0180 0.0064
pick 1058 8.2450 0.0060 0.0280 0.0078
Pooling 2436 3349.8811 0.0640 3.1720 1.3752
Convolution 2436 13800.0762 0.2260 13.2050 5.6651
CopyCPU2CPU 1218 7.0120 0.0030 0.0360 0.0058
log_softmax 1058 32.6610 0.0150 0.0550 0.0309
_backward_copy 1058 54.5520 0.0220 0.0760 0.0516
_backward_Convolution 2116 19494.2988 2.6370 12.5610 9.2128
_mul_scalar 1058 3.7320 0.0020 0.0230 0.0035
_backward_log_softmax 1058 17.7000 0.0090 0.0400 0.0167
mean 3334 63.5710 0.0080 0.0730 0.0191
_backward_mean 1058 11.4990 0.0080 0.1030 0.0109
_backward_FullyConnected 3174 593.6920 0.0280 0.5940 0.1870
WaitForVar 2276 20.1970 0.0060 0.0280 0.0089
broadcast_equal 1218 6.0310 0.0030 0.0110 0.0050
sgd_update 10580 102.5860 0.0020 0.0640 0.0097
_backward_Activation 4232 310.3530 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41620 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3660 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 529 2098.1431 0.3060 73.5850 3.9662
MXNDArrayCreateEx 610 20.8610 0.0310 0.0640 0.0342
MXNDArrayWaitAll 530 11441.3574 5.6130 24.8060 21.5875
MXNDArraySyncCopyToCPU 1140 8961.6641 0.1180 24.1980 7.8611
MXNDArrayGetDType 1140 0.4000 0.0000 0.0010 0.0004
MXNDArrayGetShape 2890 1.3220 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1220 10.7000 0.0070 0.0280 0.0088
MXNDArrayFree 13630 139.3060 0.0000 0.9140 0.0102
MXNet C API Calls 61110 548.4670 487.3580 548.4670 30.5545
MXNet C API Concurrency 122219 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1060 0.5030 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5300 2.7640 0.0000 0.0230 0.0005
MXNDArraySetGradState 5300 2.7010 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1060 0.4500 0.0000 0.0010 0.0004
MXAutogradBackwardEx 530 288.1380 0.5250 0.6810 0.5437
MXNDArrayGetContext 8460 3.6470 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17710 450.1460 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2120 5155.2671 0.7010 3.7500 2.4317
FullyConnected 3660 348.2080 0.0080 0.3430 0.0951
argmax 1220 23.8970 0.0050 0.0480 0.0196
_backward_pick 1060 16.6250 0.0130 0.0260 0.0157
_backward_mul_scalar 1060 4.5680 0.0030 0.0160 0.0043
DeleteVariable 45280 320.6070 0.0010 0.9010 0.0071
Flatten 1220 50.5830 0.0080 0.1120 0.0415
Activation 4880 221.1730 0.0030 0.3430 0.0453
SetValueOp 1060 6.7770 0.0050 0.0180 0.0064
pick 1060 8.2620 0.0060 0.0280 0.0078
Pooling 2440 3355.2351 0.0640 3.1720 1.3751
Convolution 2440 13822.4219 0.2260 13.2050 5.6649
CopyCPU2CPU 1220 7.0220 0.0030 0.0360 0.0058
log_softmax 1060 32.7350 0.0150 0.0550 0.0309
_backward_copy 1060 54.6520 0.0220 0.0760 0.0516
_backward_Convolution 2120 19531.4141 2.6370 12.5610 9.2129
_mul_scalar 1060 3.7380 0.0020 0.0230 0.0035
_backward_log_softmax 1060 17.7430 0.0090 0.0400 0.0167
mean 3340 63.6790 0.0080 0.0730 0.0191
_backward_mean 1060 11.5210 0.0080 0.1030 0.0109
_backward_FullyConnected 3180 595.1110 0.0280 0.5940 0.1871
WaitForVar 2280 20.2330 0.0060 0.0280 0.0089
broadcast_equal 1220 6.0410 0.0030 0.0110 0.0050
sgd_update 10600 102.8630 0.0020 0.0640 0.0097
_backward_Activation 4240 310.9440 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41694 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3666 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 530 2100.8979 0.3060 73.5850 3.9640
MXNDArrayCreateEx 611 20.8940 0.0310 0.0640 0.0342
MXNDArrayWaitAll 531 11463.1787 5.6130 24.8060 21.5879
MXNDArraySyncCopyToCPU 1142 8975.8984 0.1180 24.1980 7.8598
MXNDArrayGetDType 1142 0.4010 0.0000 0.0010 0.0004
MXNDArrayGetShape 2895 1.3230 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1222 10.7170 0.0070 0.0280 0.0088
MXNDArrayFree 13653 139.5780 0.0000 0.9140 0.0102
MXNet C API Calls 61217 548.5740 487.3580 548.5740 30.6080
MXNet C API Concurrency 122433 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1062 0.5040 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5310 2.7670 0.0000 0.0230 0.0005
MXNDArraySetGradState 5310 2.7060 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1062 0.4510 0.0000 0.0010 0.0004
MXAutogradBackwardEx 531 288.7010 0.5250 0.6810 0.5437
MXNDArrayGetContext 8474 3.6570 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17741 450.9460 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2124 5165.1958 0.7010 3.7500 2.4318
FullyConnected 3666 348.7370 0.0080 0.3430 0.0951
argmax 1222 23.9350 0.0050 0.0480 0.0196
_backward_pick 1062 16.6560 0.0130 0.0260 0.0157
_backward_mul_scalar 1062 4.5770 0.0030 0.0160 0.0043
DeleteVariable 45360 321.1630 0.0010 0.9010 0.0071
Flatten 1222 50.6680 0.0080 0.1120 0.0415
Activation 4888 221.5240 0.0030 0.3430 0.0453
SetValueOp 1062 6.7900 0.0050 0.0180 0.0064
pick 1062 8.2770 0.0060 0.0280 0.0078
Pooling 2444 3360.6331 0.0640 3.1720 1.3751
Convolution 2444 13844.5889 0.2260 13.2050 5.6647
CopyCPU2CPU 1222 7.0310 0.0030 0.0360 0.0058
log_softmax 1062 32.7950 0.0150 0.0550 0.0309
_backward_copy 1062 54.7560 0.0220 0.0760 0.0516
_backward_Convolution 2124 19568.2324 2.6370 12.5610 9.2129
_mul_scalar 1062 3.7450 0.0020 0.0230 0.0035
_backward_log_softmax 1062 17.7750 0.0090 0.0400 0.0167
mean 3346 63.7960 0.0080 0.0730 0.0191
_backward_mean 1062 11.5420 0.0080 0.1030 0.0109
_backward_FullyConnected 3186 596.1830 0.0280 0.5940 0.1871
WaitForVar 2284 20.2670 0.0060 0.0280 0.0089
broadcast_equal 1222 6.0500 0.0030 0.0110 0.0050
sgd_update 10620 103.0640 0.0020 0.0640 0.0097
_backward_Activation 4248 311.5130 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41768 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3672 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 531 2104.8000 0.3060 73.5850 3.9638
MXNDArrayCreateEx 612 20.9280 0.0310 0.0640 0.0342
MXNDArrayWaitAll 532 11484.8721 5.6130 24.8060 21.5881
MXNDArraySyncCopyToCPU 1144 8990.1914 0.1180 24.1980 7.8586
MXNDArrayGetDType 1144 0.4010 0.0000 0.0010 0.0004
MXNDArrayGetShape 2900 1.3240 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1224 10.7340 0.0070 0.0280 0.0088
MXNDArrayFree 13676 139.8000 0.0000 0.9140 0.0102
MXNet C API Calls 61324 548.6810 487.3580 548.6810 30.6615
MXNet C API Concurrency 122647 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1064 0.5050 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5320 2.7710 0.0000 0.0230 0.0005
MXNDArraySetGradState 5320 2.7130 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1064 0.4510 0.0000 0.0010 0.0004
MXAutogradBackwardEx 532 289.2390 0.5250 0.6810 0.5437
MXNDArrayGetContext 8488 3.6600 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17772 451.7200 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2128 5174.8979 0.7010 3.7500 2.4318
FullyConnected 3672 349.2660 0.0080 0.3430 0.0951
argmax 1224 23.9720 0.0050 0.0480 0.0196
_backward_pick 1064 16.6870 0.0130 0.0260 0.0157
_backward_mul_scalar 1064 4.5860 0.0030 0.0160 0.0043
DeleteVariable 45440 321.6960 0.0010 0.9010 0.0071
Flatten 1224 50.7480 0.0080 0.1120 0.0415
Activation 4896 221.8770 0.0030 0.3430 0.0453
SetValueOp 1064 6.8020 0.0050 0.0180 0.0064
pick 1064 8.2910 0.0060 0.0280 0.0078
Pooling 2448 3365.9761 0.0640 3.1720 1.3750
Convolution 2448 13866.6553 0.2260 13.2050 5.6645
CopyCPU2CPU 1224 7.0410 0.0030 0.0360 0.0058
log_softmax 1064 32.8550 0.0150 0.0550 0.0309
_backward_copy 1064 54.8570 0.0220 0.0760 0.0516
_backward_Convolution 2128 19604.9551 2.6370 12.5610 9.2129
_mul_scalar 1064 3.7520 0.0020 0.0230 0.0035
_backward_log_softmax 1064 17.8070 0.0090 0.0400 0.0167
mean 3352 63.9160 0.0080 0.0730 0.0191
_backward_mean 1064 11.5630 0.0080 0.1030 0.0109
_backward_FullyConnected 3192 597.2420 0.0280 0.5940 0.1871
WaitForVar 2288 20.3010 0.0060 0.0280 0.0089
broadcast_equal 1224 6.0590 0.0030 0.0110 0.0050
sgd_update 10640 103.2670 0.0020 0.0640 0.0097
_backward_Activation 4256 312.0800 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41842 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3678 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 532 2108.7141 0.3060 73.5850 3.9637
MXNDArrayCreateEx 613 20.9620 0.0310 0.0640 0.0342
MXNDArrayWaitAll 533 11506.7461 5.6130 24.8060 21.5886
MXNDArraySyncCopyToCPU 1146 9004.5420 0.1180 24.1980 7.8574
MXNDArrayGetDType 1146 0.4010 0.0000 0.0010 0.0003
MXNDArrayGetShape 2905 1.3270 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1226 10.7510 0.0070 0.0280 0.0088
MXNDArrayFree 13699 140.0300 0.0000 0.9140 0.0102
MXNet C API Calls 61431 548.7880 487.3580 548.7880 30.7150
MXNet C API Concurrency 122861 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1066 0.5050 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5330 2.7760 0.0000 0.0230 0.0005
MXNDArraySetGradState 5330 2.7180 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1066 0.4520 0.0000 0.0010 0.0004
MXAutogradBackwardEx 533 289.7770 0.5250 0.6810 0.5437
MXNDArrayGetContext 8502 3.6700 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17803 452.5040 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2132 5184.6001 0.7010 3.7500 2.4318
FullyConnected 3678 349.8170 0.0080 0.3430 0.0951
argmax 1226 24.0100 0.0050 0.0480 0.0196
_backward_pick 1066 16.7200 0.0130 0.0260 0.0157
_backward_mul_scalar 1066 4.5940 0.0030 0.0160 0.0043
DeleteVariable 45520 322.2660 0.0010 0.9010 0.0071
Flatten 1226 50.8310 0.0080 0.1120 0.0415
Activation 4904 222.2310 0.0030 0.3430 0.0453
SetValueOp 1066 6.8150 0.0050 0.0180 0.0064
pick 1066 8.3070 0.0060 0.0280 0.0078
Pooling 2452 3371.3279 0.0640 3.1720 1.3749
Convolution 2452 13888.8223 0.2260 13.2050 5.6643
CopyCPU2CPU 1226 7.0500 0.0030 0.0360 0.0058
log_softmax 1066 32.9140 0.0150 0.0550 0.0309
_backward_copy 1066 54.9610 0.0220 0.0760 0.0516
_backward_Convolution 2132 19642.0430 2.6370 12.5610 9.2130
_mul_scalar 1066 3.7600 0.0020 0.0230 0.0035
_backward_log_softmax 1066 17.8390 0.0090 0.0400 0.0167
mean 3358 64.0360 0.0080 0.0730 0.0191
_backward_mean 1066 11.5850 0.0080 0.1030 0.0109
_backward_FullyConnected 3198 598.3100 0.0280 0.5940 0.1871
WaitForVar 2292 20.3330 0.0060 0.0280 0.0089
broadcast_equal 1226 6.0690 0.0030 0.0110 0.0050
sgd_update 10660 103.4170 0.0020 0.0640 0.0097
_backward_Activation 4264 312.6670 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41916 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3684 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 533 2112.6160 0.3060 73.5850 3.9636
MXNDArrayCreateEx 614 20.9960 0.0310 0.0640 0.0342
MXNDArrayWaitAll 534 11528.6172 5.6130 24.8060 21.5892
MXNDArraySyncCopyToCPU 1148 9019.3760 0.1180 24.1980 7.8566
MXNDArrayGetDType 1148 0.4030 0.0000 0.0010 0.0004
MXNDArrayGetShape 2910 1.3300 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1228 10.7670 0.0070 0.0280 0.0088
MXNDArrayFree 13722 140.2630 0.0000 0.9140 0.0102
MXNet C API Calls 61538 548.8950 487.3580 548.8950 30.7685
MXNet C API Concurrency 123075 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1068 0.5050 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5340 2.7800 0.0000 0.0230 0.0005
MXNDArraySetGradState 5340 2.7190 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1068 0.4530 0.0000 0.0010 0.0004
MXAutogradBackwardEx 534 290.3140 0.5250 0.6810 0.5437
MXNDArrayGetContext 8516 3.6750 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17834 453.2830 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2136 5194.3569 0.7010 3.7500 2.4318
FullyConnected 3684 350.5900 0.0080 0.3430 0.0952
argmax 1228 24.0540 0.0050 0.0480 0.0196
_backward_pick 1068 16.7520 0.0130 0.0260 0.0157
_backward_mul_scalar 1068 4.6030 0.0030 0.0160 0.0043
DeleteVariable 45600 322.8100 0.0010 0.9010 0.0071
Flatten 1228 50.9100 0.0080 0.1120 0.0415
Activation 4912 222.6200 0.0030 0.3430 0.0453
SetValueOp 1068 6.8270 0.0050 0.0180 0.0064
pick 1068 8.3220 0.0060 0.0280 0.0078
Pooling 2456 3376.7251 0.0640 3.1720 1.3749
Convolution 2456 13911.1738 0.2260 13.2050 5.6642
CopyCPU2CPU 1228 7.0600 0.0030 0.0360 0.0057
log_softmax 1068 32.9870 0.0150 0.0550 0.0309
_backward_copy 1068 55.0610 0.0220 0.0760 0.0516
_backward_Convolution 2136 19679.0898 2.6370 12.5610 9.2131
_mul_scalar 1068 3.7670 0.0020 0.0230 0.0035
_backward_log_softmax 1068 17.8820 0.0090 0.0400 0.0167
mean 3364 64.1420 0.0080 0.0730 0.0191
_backward_mean 1068 11.6060 0.0080 0.1030 0.0109
_backward_FullyConnected 3204 599.7310 0.0280 0.5940 0.1872
WaitForVar 2296 20.3700 0.0060 0.0280 0.0089
broadcast_equal 1228 6.0780 0.0030 0.0110 0.0049
sgd_update 10680 103.6340 0.0020 0.0640 0.0097
_backward_Activation 4272 313.3440 0.0070 0.3900 0.0733
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 41990 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3690 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 534 2116.5530 0.3060 73.5850 3.9636
MXNDArrayCreateEx 615 21.0300 0.0310 0.0640 0.0342
MXNDArrayWaitAll 535 11550.4531 5.6130 24.8060 21.5896
MXNDArraySyncCopyToCPU 1150 9033.7568 0.1180 24.1980 7.8554
MXNDArrayGetDType 1150 0.4050 0.0000 0.0010 0.0004
MXNDArrayGetShape 2915 1.3330 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1230 10.7830 0.0070 0.0280 0.0088
MXNDArrayFree 13745 140.4900 0.0000 0.9140 0.0102
MXNet C API Calls 61645 549.0020 487.3580 549.0020 30.8220
MXNet C API Concurrency 123289 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1070 0.5070 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5350 2.7830 0.0000 0.0230 0.0005
MXNDArraySetGradState 5350 2.7260 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1070 0.4550 0.0000 0.0010 0.0004
MXAutogradBackwardEx 535 290.8460 0.5250 0.6810 0.5436
MXNDArrayGetContext 8530 3.6830 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17865 454.0570 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2140 5204.2192 0.7010 3.7500 2.4319
FullyConnected 3690 351.1120 0.0080 0.3430 0.0952
argmax 1230 24.0930 0.0050 0.0480 0.0196
_backward_pick 1070 16.7830 0.0130 0.0260 0.0157
_backward_mul_scalar 1070 4.6110 0.0030 0.0160 0.0043
DeleteVariable 45680 323.3390 0.0010 0.9010 0.0071
Flatten 1230 50.9890 0.0080 0.1120 0.0415
Activation 4920 222.9720 0.0030 0.3430 0.0453
SetValueOp 1070 6.8390 0.0050 0.0180 0.0064
pick 1070 8.3360 0.0060 0.0280 0.0078
Pooling 2460 3382.0540 0.0640 3.1720 1.3748
Convolution 2460 13933.3711 0.2260 13.2050 5.6640
CopyCPU2CPU 1230 7.0690 0.0030 0.0360 0.0057
log_softmax 1070 33.0470 0.0150 0.0550 0.0309
_backward_copy 1070 55.1650 0.0220 0.0760 0.0516
_backward_Convolution 2140 19715.9805 2.6370 12.5610 9.2131
_mul_scalar 1070 3.7750 0.0020 0.0230 0.0035
_backward_log_softmax 1070 17.9130 0.0090 0.0400 0.0167
mean 3370 64.2610 0.0080 0.0730 0.0191
_backward_mean 1070 11.6270 0.0080 0.1030 0.0109
_backward_FullyConnected 3210 600.7950 0.0280 0.5940 0.1872
WaitForVar 2300 20.4040 0.0060 0.0280 0.0089
broadcast_equal 1230 6.0880 0.0030 0.0110 0.0049
sgd_update 10700 103.7880 0.0020 0.0640 0.0097
_backward_Activation 4280 313.9960 0.0070 0.3900 0.0734
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42064 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3696 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 535 2120.4709 0.3060 73.5850 3.9635
MXNDArrayCreateEx 616 21.0640 0.0310 0.0640 0.0342
MXNDArrayWaitAll 536 11572.1191 5.6130 24.8060 21.5898
MXNDArraySyncCopyToCPU 1152 9048.0684 0.1180 24.1980 7.8542
MXNDArrayGetDType 1152 0.4060 0.0000 0.0010 0.0004
MXNDArrayGetShape 2920 1.3360 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1232 10.8010 0.0070 0.0280 0.0088
MXNDArrayFree 13768 140.7230 0.0000 0.9140 0.0102
MXNet C API Calls 61752 549.1090 487.3580 549.1090 30.8755
MXNet C API Concurrency 123503 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1072 0.5070 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5360 2.7850 0.0000 0.0230 0.0005
MXNDArraySetGradState 5360 2.7320 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1072 0.4550 0.0000 0.0010 0.0004
MXAutogradBackwardEx 536 291.3920 0.5250 0.6810 0.5436
MXNDArrayGetContext 8544 3.6910 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17896 454.8390 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2144 5213.9580 0.7010 3.7500 2.4319
FullyConnected 3696 351.6370 0.0080 0.3430 0.0951
argmax 1232 24.1300 0.0050 0.0480 0.0196
_backward_pick 1072 16.8150 0.0130 0.0260 0.0157
_backward_mul_scalar 1072 4.6200 0.0030 0.0160 0.0043
DeleteVariable 45760 323.8770 0.0010 0.9010 0.0071
Flatten 1232 51.0730 0.0080 0.1120 0.0415
Activation 4928 223.3280 0.0030 0.3430 0.0453
SetValueOp 1072 6.8510 0.0050 0.0180 0.0064
pick 1072 8.3500 0.0060 0.0280 0.0078
Pooling 2464 3387.4121 0.0640 3.1720 1.3748
Convolution 2464 13955.4639 0.2260 13.2050 5.6637
CopyCPU2CPU 1232 7.0790 0.0030 0.0360 0.0057
log_softmax 1072 33.1070 0.0150 0.0550 0.0309
_backward_copy 1072 55.2690 0.0220 0.0760 0.0516
_backward_Convolution 2144 19752.6641 2.6370 12.5610 9.2130
_mul_scalar 1072 3.7830 0.0020 0.0230 0.0035
_backward_log_softmax 1072 17.9450 0.0090 0.0400 0.0167
mean 3376 64.3770 0.0080 0.0730 0.0191
_backward_mean 1072 11.6470 0.0080 0.1030 0.0109
_backward_FullyConnected 3216 601.8660 0.0280 0.5940 0.1871
WaitForVar 2304 20.4370 0.0060 0.0280 0.0089
broadcast_equal 1232 6.0980 0.0030 0.0110 0.0049
sgd_update 10720 103.9890 0.0020 0.0640 0.0097
_backward_Activation 4288 314.5330 0.0070 0.3900 0.0734
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42138 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3702 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 536 2124.3750 0.3060 73.5850 3.9634
MXNDArrayCreateEx 617 21.0970 0.0310 0.0640 0.0342
MXNDArrayWaitAll 537 11593.9629 5.6130 24.8060 21.5902
MXNDArraySyncCopyToCPU 1154 9062.4443 0.1180 24.1980 7.8531
MXNDArrayGetDType 1154 0.4070 0.0000 0.0010 0.0004
MXNDArrayGetShape 2925 1.3380 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1234 10.8190 0.0070 0.0280 0.0088
MXNDArrayFree 13791 140.9700 0.0000 0.9140 0.0102
MXNet C API Calls 61859 549.2160 487.3580 549.2160 30.9290
MXNet C API Concurrency 123717 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1074 0.5070 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5370 2.7900 0.0000 0.0230 0.0005
MXNDArraySetGradState 5370 2.7350 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1074 0.4560 0.0000 0.0010 0.0004
MXAutogradBackwardEx 537 291.9310 0.5250 0.6810 0.5436
MXNDArrayGetContext 8558 3.6960 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17927 455.6130 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2148 5223.6338 0.7010 3.7500 2.4319
FullyConnected 3702 352.1610 0.0080 0.3430 0.0951
argmax 1234 24.1690 0.0050 0.0480 0.0196
_backward_pick 1074 16.8470 0.0130 0.0260 0.0157
_backward_mul_scalar 1074 4.6290 0.0030 0.0160 0.0043
DeleteVariable 45840 324.3920 0.0010 0.9010 0.0071
Flatten 1234 51.1530 0.0080 0.1120 0.0415
Activation 4936 223.7220 0.0030 0.3430 0.0453
SetValueOp 1074 6.8650 0.0050 0.0180 0.0064
pick 1074 8.3650 0.0060 0.0280 0.0078
Pooling 2468 3392.7581 0.0640 3.1720 1.3747
Convolution 2468 13977.6494 0.2260 13.2050 5.6636
CopyCPU2CPU 1234 7.0890 0.0030 0.0360 0.0057
log_softmax 1074 33.1670 0.0150 0.0550 0.0309
_backward_copy 1074 55.3710 0.0220 0.0760 0.0516
_backward_Convolution 2148 19789.6777 2.6370 12.5610 9.2131
_mul_scalar 1074 3.7900 0.0020 0.0230 0.0035
_backward_log_softmax 1074 17.9770 0.0090 0.0400 0.0167
mean 3382 64.4910 0.0080 0.0730 0.0191
_backward_mean 1074 11.6680 0.0080 0.1030 0.0109
_backward_FullyConnected 3222 602.9300 0.0280 0.5940 0.1871
WaitForVar 2308 20.4720 0.0060 0.0280 0.0089
broadcast_equal 1234 6.1080 0.0030 0.0110 0.0049
sgd_update 10740 104.1420 0.0020 0.0640 0.0097
_backward_Activation 4296 315.1730 0.0070 0.3900 0.0734
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42212 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3708 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 537 2128.2700 0.3060 73.5850 3.9633
MXNDArrayCreateEx 618 21.1310 0.0310 0.0640 0.0342
MXNDArrayWaitAll 538 11615.8291 5.6130 24.8060 21.5908
MXNDArraySyncCopyToCPU 1156 9077.3740 0.1180 24.1980 7.8524
MXNDArrayGetDType 1156 0.4070 0.0000 0.0010 0.0004
MXNDArrayGetShape 2930 1.3400 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1236 10.8360 0.0070 0.0280 0.0088
MXNDArrayFree 13814 141.1980 0.0000 0.9140 0.0102
MXNet C API Calls 61966 549.3230 487.3580 549.3230 30.9825
MXNet C API Concurrency 123931 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1076 0.5070 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5380 2.7960 0.0000 0.0230 0.0005
MXNDArraySetGradState 5380 2.7380 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1076 0.4580 0.0000 0.0010 0.0004
MXAutogradBackwardEx 538 292.4920 0.5250 0.6810 0.5437
MXNDArrayGetContext 8572 3.7000 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17958 456.3870 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2152 5233.5889 0.7010 3.7500 2.4320
FullyConnected 3708 352.9600 0.0080 0.3430 0.0952
argmax 1236 24.2150 0.0050 0.0480 0.0196
_backward_pick 1076 16.8790 0.0130 0.0260 0.0157
_backward_mul_scalar 1076 4.6380 0.0030 0.0160 0.0043
DeleteVariable 45920 324.9420 0.0010 0.9010 0.0071
Flatten 1236 51.2320 0.0080 0.1120 0.0414
Activation 4944 224.1020 0.0030 0.3430 0.0453
SetValueOp 1076 6.8780 0.0050 0.0180 0.0064
pick 1076 8.3800 0.0060 0.0280 0.0078
Pooling 2472 3398.1541 0.0640 3.1720 1.3747
Convolution 2472 13999.9629 0.2260 13.2050 5.6634
CopyCPU2CPU 1236 7.0990 0.0030 0.0360 0.0057
log_softmax 1076 33.2400 0.0150 0.0550 0.0309
_backward_copy 1076 55.4720 0.0220 0.0760 0.0516
_backward_Convolution 2152 19826.7031 2.6370 12.5610 9.2132
_mul_scalar 1076 3.7970 0.0020 0.0230 0.0035
_backward_log_softmax 1076 18.0180 0.0090 0.0400 0.0167
mean 3388 64.5980 0.0080 0.0730 0.0191
_backward_mean 1076 11.6890 0.0080 0.1030 0.0109
_backward_FullyConnected 3228 604.3410 0.0280 0.5940 0.1872
WaitForVar 2312 20.5060 0.0060 0.0280 0.0089
broadcast_equal 1236 6.1180 0.0030 0.0110 0.0049
sgd_update 10760 104.3540 0.0020 0.0640 0.0097
_backward_Activation 4304 315.8440 0.0070 0.3900 0.0734
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42286 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3714 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 538 2132.1709 0.3060 73.5850 3.9631
MXNDArrayCreateEx 619 21.1640 0.0310 0.0640 0.0342
MXNDArrayWaitAll 539 11637.5732 5.6130 24.8060 21.5910
MXNDArraySyncCopyToCPU 1158 9091.8799 0.1180 24.1980 7.8514
MXNDArrayGetDType 1158 0.4090 0.0000 0.0010 0.0004
MXNDArrayGetShape 2935 1.3440 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1238 10.8530 0.0070 0.0280 0.0088
MXNDArrayFree 13837 141.4200 0.0000 0.9140 0.0102
MXNet C API Calls 62073 549.4300 487.3580 549.4300 31.0360
MXNet C API Concurrency 124145 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1078 0.5090 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5390 2.8010 0.0000 0.0230 0.0005
MXNDArraySetGradState 5390 2.7410 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1078 0.4600 0.0000 0.0010 0.0004
MXAutogradBackwardEx 539 293.0240 0.5250 0.6810 0.5436
MXNDArrayGetContext 8586 3.7080 0.0000 0.0070 0.0004
MXImperativeInvokeEx 17989 457.1610 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2156 5243.3408 0.7010 3.7500 2.4320
FullyConnected 3714 353.5630 0.0080 0.3430 0.0952
argmax 1238 24.2520 0.0050 0.0480 0.0196
_backward_pick 1078 16.9100 0.0130 0.0260 0.0157
_backward_mul_scalar 1078 4.6460 0.0030 0.0160 0.0043
DeleteVariable 46000 325.4470 0.0010 0.9010 0.0071
Flatten 1238 51.3110 0.0080 0.1120 0.0414
Activation 4952 224.4860 0.0030 0.3430 0.0453
SetValueOp 1078 6.8890 0.0050 0.0180 0.0064
pick 1078 8.3950 0.0060 0.0280 0.0078
Pooling 2476 3403.5061 0.0640 3.1720 1.3746
Convolution 2476 14022.1211 0.2260 13.2050 5.6632
CopyCPU2CPU 1238 7.1080 0.0030 0.0360 0.0057
log_softmax 1078 33.3010 0.0150 0.0550 0.0309
_backward_copy 1078 55.5750 0.0220 0.0760 0.0516
_backward_Convolution 2156 19863.4922 2.6370 12.5610 9.2131
_mul_scalar 1078 3.8040 0.0020 0.0230 0.0035
_backward_log_softmax 1078 18.0580 0.0090 0.0400 0.0168
mean 3394 64.7060 0.0080 0.0730 0.0191
_backward_mean 1078 11.7100 0.0080 0.1030 0.0109
_backward_FullyConnected 3234 605.4760 0.0280 0.5940 0.1872
WaitForVar 2316 20.5400 0.0060 0.0280 0.0089
broadcast_equal 1238 6.1280 0.0030 0.0110 0.0049
sgd_update 10780 104.5900 0.0020 0.0640 0.0097
_backward_Activation 4312 316.4260 0.0070 0.3900 0.0734
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42360 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3720 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 539 2136.0801 0.3060 73.5850 3.9630
MXNDArrayCreateEx 620 21.1990 0.0310 0.0640 0.0342
MXNDArrayWaitAll 540 11662.1816 5.6130 24.8060 21.5966
MXNDArraySyncCopyToCPU 1160 9109.7939 0.1180 24.1980 7.8533
MXNDArrayGetDType 1160 0.4090 0.0000 0.0010 0.0004
MXNDArrayGetShape 2940 1.3440 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1240 10.8700 0.0070 0.0280 0.0088
MXNDArrayFree 13860 141.6450 0.0000 0.9140 0.0102
MXNet C API Calls 62180 549.5370 487.3580 549.5370 31.0895
MXNet C API Concurrency 124359 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1080 0.5110 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5400 2.8080 0.0000 0.0230 0.0005
MXNDArraySetGradState 5400 2.7480 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1080 0.4600 0.0000 0.0010 0.0004
MXAutogradBackwardEx 540 293.5900 0.5250 0.6810 0.5437
MXNDArrayGetContext 8600 3.7180 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18020 457.9420 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2160 5255.5508 0.7010 3.7500 2.4331
FullyConnected 3720 354.4650 0.0080 0.3430 0.0953
argmax 1240 24.3020 0.0050 0.0480 0.0196
_backward_pick 1080 16.9450 0.0130 0.0260 0.0157
_backward_mul_scalar 1080 4.6550 0.0030 0.0160 0.0043
DeleteVariable 46080 326.0320 0.0010 0.9010 0.0071
Flatten 1240 51.4130 0.0080 0.1120 0.0415
Activation 4960 225.0930 0.0030 0.3430 0.0454
SetValueOp 1080 6.9050 0.0050 0.0180 0.0064
pick 1080 8.4140 0.0060 0.0280 0.0078
Pooling 2480 3412.9729 0.0640 3.1720 1.3762
Convolution 2480 14044.3770 0.2260 13.2050 5.6631
CopyCPU2CPU 1240 7.1190 0.0030 0.0360 0.0057
log_softmax 1080 33.3790 0.0150 0.0550 0.0309
_backward_copy 1080 55.6730 0.0220 0.0760 0.0515
_backward_Convolution 2160 19904.4277 2.6370 12.5610 9.2150
_mul_scalar 1080 3.8130 0.0020 0.0230 0.0035
_backward_log_softmax 1080 18.1040 0.0090 0.0400 0.0168
mean 3400 64.8310 0.0080 0.0730 0.0191
_backward_mean 1080 11.7370 0.0080 0.1030 0.0109
_backward_FullyConnected 3240 607.2040 0.0280 0.5940 0.1874
WaitForVar 2320 20.5810 0.0060 0.0280 0.0089
broadcast_equal 1240 6.1410 0.0030 0.0110 0.0050
sgd_update 10800 104.8870 0.0020 0.0640 0.0097
_backward_Activation 4320 317.2430 0.0070 0.3900 0.0734
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42434 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3726 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 540 2139.9810 0.3060 73.5850 3.9629
MXNDArrayCreateEx 621 21.2320 0.0310 0.0640 0.0342
MXNDArrayWaitAll 541 11686.9072 5.6130 24.8060 21.6024
MXNDArraySyncCopyToCPU 1162 9127.5947 0.1180 24.1980 7.8551
MXNDArrayGetDType 1162 0.4100 0.0000 0.0010 0.0004
MXNDArrayGetShape 2945 1.3470 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1242 10.8880 0.0070 0.0280 0.0088
MXNDArrayFree 13883 141.8700 0.0000 0.9140 0.0102
MXNet C API Calls 62287 549.6440 487.3580 549.6440 31.1430
MXNet C API Concurrency 124573 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1082 0.5130 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5410 2.8140 0.0000 0.0230 0.0005
MXNDArraySetGradState 5410 2.7520 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1082 0.4620 0.0000 0.0010 0.0004
MXAutogradBackwardEx 541 294.1520 0.5250 0.6810 0.5437
MXNDArrayGetContext 8614 3.7210 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18051 458.7150 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2164 5267.7861 0.7010 3.7500 2.4343
FullyConnected 3726 355.3590 0.0080 0.3430 0.0954
argmax 1242 24.3530 0.0050 0.0480 0.0196
_backward_pick 1082 16.9810 0.0130 0.0260 0.0157
_backward_mul_scalar 1082 4.6640 0.0030 0.0160 0.0043
DeleteVariable 46160 326.6040 0.0010 0.9010 0.0071
Flatten 1242 51.4930 0.0080 0.1120 0.0415
Activation 4968 225.6260 0.0030 0.3430 0.0454
SetValueOp 1082 6.9190 0.0050 0.0180 0.0064
pick 1082 8.4330 0.0060 0.0280 0.0078
Pooling 2484 3422.3979 0.0640 3.1720 1.3778
Convolution 2484 14066.5225 0.2260 13.2050 5.6629
CopyCPU2CPU 1242 7.1310 0.0030 0.0360 0.0057
log_softmax 1082 33.4560 0.0150 0.0550 0.0309
_backward_copy 1082 55.7710 0.0220 0.0760 0.0515
_backward_Convolution 2164 19945.6230 2.6370 12.5610 9.2170
_mul_scalar 1082 3.8220 0.0020 0.0230 0.0035
_backward_log_softmax 1082 18.1500 0.0090 0.0400 0.0168
mean 3406 64.9590 0.0080 0.0730 0.0191
_backward_mean 1082 11.7630 0.0080 0.1030 0.0109
_backward_FullyConnected 3246 608.9260 0.0280 0.5940 0.1876
WaitForVar 2324 20.6220 0.0060 0.0280 0.0089
broadcast_equal 1242 6.1540 0.0030 0.0110 0.0050
sgd_update 10820 105.1860 0.0020 0.0640 0.0097
_backward_Activation 4328 318.0530 0.0070 0.3900 0.0735
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42508 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3732 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 541 2143.8850 0.3060 73.5850 3.9628
MXNDArrayCreateEx 622 21.2670 0.0310 0.0640 0.0342
MXNDArrayWaitAll 542 11711.5801 5.6130 24.8060 21.6081
MXNDArraySyncCopyToCPU 1164 9145.4326 0.1180 24.1980 7.8569
MXNDArrayGetDType 1164 0.4100 0.0000 0.0010 0.0004
MXNDArrayGetShape 2950 1.3500 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1244 10.9050 0.0070 0.0280 0.0088
MXNDArrayFree 13906 142.0990 0.0000 0.9140 0.0102
MXNet C API Calls 62394 549.7510 487.3580 549.7510 31.1965
MXNet C API Concurrency 124787 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1084 0.5140 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5420 2.8180 0.0000 0.0230 0.0005
MXNDArraySetGradState 5420 2.7570 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1084 0.4620 0.0000 0.0010 0.0004
MXAutogradBackwardEx 542 294.7140 0.5250 0.6810 0.5438
MXNDArrayGetContext 8628 3.7290 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18082 459.4890 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2168 5279.8652 0.7010 3.7500 2.4354
FullyConnected 3732 356.2300 0.0080 0.3430 0.0955
argmax 1244 24.4030 0.0050 0.0480 0.0196
_backward_pick 1084 17.0170 0.0130 0.0260 0.0157
_backward_mul_scalar 1084 4.6720 0.0030 0.0160 0.0043
DeleteVariable 46240 327.1830 0.0010 0.9010 0.0071
Flatten 1244 51.5740 0.0080 0.1120 0.0415
Activation 4976 226.1580 0.0030 0.3430 0.0454
SetValueOp 1084 6.9340 0.0050 0.0180 0.0064
pick 1084 8.4520 0.0060 0.0280 0.0078
Pooling 2488 3431.8401 0.0640 3.1720 1.3794
Convolution 2488 14088.7686 0.2260 13.2050 5.6627
CopyCPU2CPU 1244 7.1420 0.0030 0.0360 0.0057
log_softmax 1084 33.5340 0.0150 0.0550 0.0309
_backward_copy 1084 55.8690 0.0220 0.0760 0.0515
_backward_Convolution 2168 19986.7832 2.6370 12.5610 9.2190
_mul_scalar 1084 3.8300 0.0020 0.0230 0.0035
_backward_log_softmax 1084 18.1970 0.0090 0.0400 0.0168
mean 3412 65.1080 0.0080 0.0730 0.0191
_backward_mean 1084 11.7890 0.0080 0.1030 0.0109
_backward_FullyConnected 3252 610.6450 0.0280 0.5940 0.1878
WaitForVar 2328 20.6630 0.0060 0.0280 0.0089
broadcast_equal 1244 6.1670 0.0030 0.0110 0.0050
sgd_update 10840 105.4940 0.0020 0.0640 0.0097
_backward_Activation 4336 318.8610 0.0070 0.3900 0.0735
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42582 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3738 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 542 2147.7920 0.3060 73.5850 3.9627
MXNDArrayCreateEx 623 21.3010 0.0310 0.0640 0.0342
MXNDArrayWaitAll 543 11736.1533 5.6130 24.8060 21.6135
MXNDArraySyncCopyToCPU 1166 9163.2285 0.1180 24.1980 7.8587
MXNDArrayGetDType 1166 0.4100 0.0000 0.0010 0.0004
MXNDArrayGetShape 2955 1.3520 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1246 10.9240 0.0070 0.0280 0.0088
MXNDArrayFree 13929 142.3220 0.0000 0.9140 0.0102
MXNet C API Calls 62501 549.8580 487.3580 549.8580 31.2500
MXNet C API Concurrency 125001 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1086 0.5160 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5430 2.8230 0.0000 0.0230 0.0005
MXNDArraySetGradState 5430 2.7590 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1086 0.4640 0.0000 0.0010 0.0004
MXAutogradBackwardEx 543 295.2480 0.5250 0.6810 0.5437
MXNDArrayGetContext 8642 3.7370 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18113 460.2640 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2172 5292.0078 0.7010 3.7500 2.4365
FullyConnected 3738 357.1430 0.0080 0.3430 0.0955
argmax 1246 24.4530 0.0050 0.0480 0.0196
_backward_pick 1086 17.0510 0.0130 0.0260 0.0157
_backward_mul_scalar 1086 4.6800 0.0030 0.0160 0.0043
DeleteVariable 46320 327.7220 0.0010 0.9010 0.0071
Flatten 1246 51.6530 0.0080 0.1120 0.0415
Activation 4984 226.6910 0.0030 0.3430 0.0455
SetValueOp 1086 6.9480 0.0050 0.0180 0.0064
pick 1086 8.4710 0.0060 0.0280 0.0078
Pooling 2492 3441.2871 0.0640 3.1720 1.3809
Convolution 2492 14110.9170 0.2260 13.2050 5.6625
CopyCPU2CPU 1246 7.1540 0.0030 0.0360 0.0057
log_softmax 1086 33.6100 0.0150 0.0550 0.0309
_backward_copy 1086 55.9650 0.0220 0.0760 0.0515
_backward_Convolution 2172 20027.7227 2.6370 12.5610 9.2209
_mul_scalar 1086 3.8380 0.0020 0.0230 0.0035
_backward_log_softmax 1086 18.2440 0.0090 0.0400 0.0168
mean 3418 65.2360 0.0080 0.0730 0.0191
_backward_mean 1086 11.8150 0.0080 0.1030 0.0109
_backward_FullyConnected 3258 612.3730 0.0280 0.5940 0.1880
WaitForVar 2332 20.7040 0.0060 0.0280 0.0089
broadcast_equal 1246 6.1800 0.0030 0.0110 0.0050
sgd_update 10860 105.7940 0.0020 0.0640 0.0097
_backward_Activation 4344 319.6690 0.0070 0.3900 0.0736
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42656 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3744 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 543 2151.6960 0.3060 73.5850 3.9626
MXNDArrayCreateEx 624 21.3350 0.0310 0.0640 0.0342
MXNDArrayWaitAll 544 11760.6641 5.6130 24.8060 21.6189
MXNDArraySyncCopyToCPU 1168 9180.9072 0.1180 24.1980 7.8604
MXNDArrayGetDType 1168 0.4110 0.0000 0.0010 0.0004
MXNDArrayGetShape 2960 1.3540 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1248 10.9410 0.0070 0.0280 0.0088
MXNDArrayFree 13952 142.5460 0.0000 0.9140 0.0102
MXNet C API Calls 62608 549.9650 487.3580 549.9650 31.3035
MXNet C API Concurrency 125215 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1088 0.5160 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5440 2.8300 0.0000 0.0230 0.0005
MXNDArraySetGradState 5440 2.7630 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1088 0.4640 0.0000 0.0010 0.0004
MXAutogradBackwardEx 544 295.7850 0.5250 0.6810 0.5437
MXNDArrayGetContext 8656 3.7450 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18144 461.0420 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2176 5304.0469 0.7010 3.7500 2.4375
FullyConnected 3744 358.0400 0.0080 0.3430 0.0956
argmax 1248 24.5030 0.0050 0.0480 0.0196
_backward_pick 1088 17.0850 0.0130 0.0260 0.0157
_backward_mul_scalar 1088 4.6880 0.0030 0.0160 0.0043
DeleteVariable 46400 328.2820 0.0010 0.9010 0.0071
Flatten 1248 51.7330 0.0080 0.1120 0.0415
Activation 4992 227.2240 0.0030 0.3430 0.0455
SetValueOp 1088 6.9640 0.0050 0.0180 0.0064
pick 1088 8.4890 0.0060 0.0280 0.0078
Pooling 2496 3450.6589 0.0640 3.1720 1.3825
Convolution 2496 14133.0020 0.2260 13.2050 5.6623
CopyCPU2CPU 1248 7.1640 0.0030 0.0360 0.0057
log_softmax 1088 33.6860 0.0150 0.0550 0.0310
_backward_copy 1088 56.0620 0.0220 0.0760 0.0515
_backward_Convolution 2176 20068.5020 2.6370 12.5610 9.2227
_mul_scalar 1088 3.8460 0.0020 0.0230 0.0035
_backward_log_softmax 1088 18.2900 0.0090 0.0400 0.0168
mean 3424 65.3590 0.0080 0.0730 0.0191
_backward_mean 1088 11.8400 0.0080 0.1030 0.0109
_backward_FullyConnected 3264 614.0620 0.0280 0.5940 0.1881
WaitForVar 2336 20.7460 0.0060 0.0280 0.0089
broadcast_equal 1248 6.1930 0.0030 0.0110 0.0050
sgd_update 10880 106.1960 0.0020 0.0790 0.0098
_backward_Activation 4352 320.4700 0.0070 0.3900 0.0736
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42730 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3750 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 544 2155.6069 0.3060 73.5850 3.9625
MXNDArrayCreateEx 625 21.3690 0.0310 0.0640 0.0342
MXNDArrayWaitAll 545 11785.3057 5.6130 24.8060 21.6244
MXNDArraySyncCopyToCPU 1170 9198.7461 0.1180 24.1980 7.8622
MXNDArrayGetDType 1170 0.4120 0.0000 0.0010 0.0004
MXNDArrayGetShape 2965 1.3580 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1250 10.9590 0.0070 0.0280 0.0088
MXNDArrayFree 13975 142.7750 0.0000 0.9140 0.0102
MXNet C API Calls 62715 550.0720 487.3580 550.0720 31.3570
MXNet C API Concurrency 125429 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1090 0.5170 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5450 2.8360 0.0000 0.0230 0.0005
MXNDArraySetGradState 5450 2.7670 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1090 0.4650 0.0000 0.0010 0.0004
MXAutogradBackwardEx 545 296.3180 0.5250 0.6810 0.5437
MXNDArrayGetContext 8670 3.7540 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18175 461.8450 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2180 5316.1968 0.7010 3.7500 2.4386
FullyConnected 3750 358.9450 0.0080 0.3430 0.0957
argmax 1250 24.5530 0.0050 0.0480 0.0196
_backward_pick 1090 17.1200 0.0130 0.0260 0.0157
_backward_mul_scalar 1090 4.6960 0.0030 0.0160 0.0043
DeleteVariable 46480 328.8360 0.0010 0.9010 0.0071
Flatten 1250 51.8140 0.0080 0.1120 0.0415
Activation 5000 227.7640 0.0030 0.3430 0.0456
SetValueOp 1090 6.9780 0.0050 0.0180 0.0064
pick 1090 8.5080 0.0060 0.0280 0.0078
Pooling 2500 3460.0601 0.0640 3.1720 1.3840
Convolution 2500 14155.2188 0.2260 13.2050 5.6621
CopyCPU2CPU 1250 7.1750 0.0030 0.0360 0.0057
log_softmax 1090 33.7630 0.0150 0.0550 0.0310
_backward_copy 1090 56.1590 0.0220 0.0760 0.0515
_backward_Convolution 2180 20109.6230 2.6370 12.5610 9.2246
_mul_scalar 1090 3.8550 0.0020 0.0230 0.0035
_backward_log_softmax 1090 18.3370 0.0090 0.0400 0.0168
mean 3430 65.4840 0.0080 0.0730 0.0191
_backward_mean 1090 11.8660 0.0080 0.1030 0.0109
_backward_FullyConnected 3270 615.7540 0.0280 0.5940 0.1883
WaitForVar 2340 20.7860 0.0060 0.0280 0.0089
broadcast_equal 1250 6.2060 0.0030 0.0110 0.0050
sgd_update 10900 106.4960 0.0020 0.0790 0.0098
_backward_Activation 4360 321.2730 0.0070 0.3900 0.0737
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42804 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3756 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 545 2159.5190 0.3060 73.5850 3.9624
MXNDArrayCreateEx 626 21.4030 0.0310 0.0640 0.0342
MXNDArrayWaitAll 546 11809.9170 5.6130 24.8060 21.6299
MXNDArraySyncCopyToCPU 1172 9216.5381 0.1180 24.1980 7.8639
MXNDArrayGetDType 1172 0.4130 0.0000 0.0010 0.0004
MXNDArrayGetShape 2970 1.3610 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1252 10.9760 0.0070 0.0280 0.0088
MXNDArrayFree 13998 142.9980 0.0000 0.9140 0.0102
MXNet C API Calls 62822 550.1790 487.3580 550.1790 31.4105
MXNet C API Concurrency 125643 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1092 0.5180 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5460 2.8420 0.0000 0.0230 0.0005
MXNDArraySetGradState 5460 2.7730 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1092 0.4660 0.0000 0.0010 0.0004
MXAutogradBackwardEx 546 296.8500 0.5250 0.6810 0.5437
MXNDArrayGetContext 8684 3.7630 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18206 462.6200 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2184 5328.2710 0.7010 3.7500 2.4397
FullyConnected 3756 359.8370 0.0080 0.3430 0.0958
argmax 1252 24.6020 0.0050 0.0480 0.0197
_backward_pick 1092 17.1560 0.0130 0.0260 0.0157
_backward_mul_scalar 1092 4.7050 0.0030 0.0160 0.0043
DeleteVariable 46560 329.3990 0.0010 0.9010 0.0071
Flatten 1252 51.8940 0.0080 0.1120 0.0414
Activation 5008 228.3230 0.0030 0.3430 0.0456
SetValueOp 1092 6.9930 0.0050 0.0180 0.0064
pick 1092 8.5270 0.0060 0.0280 0.0078
Pooling 2504 3469.4719 0.0640 3.1720 1.3856
Convolution 2504 14177.4521 0.2260 13.2050 5.6619
CopyCPU2CPU 1252 7.1850 0.0030 0.0360 0.0057
log_softmax 1092 33.8400 0.0150 0.0550 0.0310
_backward_copy 1092 56.2550 0.0220 0.0760 0.0515
_backward_Convolution 2184 20150.6562 2.6370 12.5610 9.2265
_mul_scalar 1092 3.8640 0.0020 0.0230 0.0035
_backward_log_softmax 1092 18.3840 0.0090 0.0400 0.0168
mean 3436 65.6300 0.0080 0.0730 0.0191
_backward_mean 1092 11.8910 0.0080 0.1030 0.0109
_backward_FullyConnected 3276 617.4350 0.0280 0.5940 0.1885
WaitForVar 2344 20.8270 0.0060 0.0280 0.0089
broadcast_equal 1252 6.2190 0.0030 0.0110 0.0050
sgd_update 10920 106.7930 0.0020 0.0790 0.0098
_backward_Activation 4368 322.0750 0.0070 0.3900 0.0737
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42878 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3762 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 546 2163.4131 0.3060 73.5850 3.9623
MXNDArrayCreateEx 627 21.4380 0.0310 0.0640 0.0342
MXNDArrayWaitAll 547 11834.4336 5.6130 24.8060 21.6352
MXNDArraySyncCopyToCPU 1174 9234.3467 0.1180 24.1980 7.8657
MXNDArrayGetDType 1174 0.4150 0.0000 0.0010 0.0004
MXNDArrayGetShape 2975 1.3620 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1254 10.9930 0.0070 0.0280 0.0088
MXNDArrayFree 14021 143.2190 0.0000 0.9140 0.0102
MXNet C API Calls 62929 550.2860 487.3580 550.2860 31.4640
MXNet C API Concurrency 125857 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1094 0.5190 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5470 2.8460 0.0000 0.0230 0.0005
MXNDArraySetGradState 5470 2.7800 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1094 0.4670 0.0000 0.0010 0.0004
MXAutogradBackwardEx 547 297.3810 0.5250 0.6810 0.5437
MXNDArrayGetContext 8698 3.7680 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18237 463.3920 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2188 5340.3481 0.7010 3.7500 2.4407
FullyConnected 3762 360.7100 0.0080 0.3430 0.0959
argmax 1254 24.6510 0.0050 0.0480 0.0197
_backward_pick 1094 17.1900 0.0130 0.0260 0.0157
_backward_mul_scalar 1094 4.7140 0.0030 0.0160 0.0043
DeleteVariable 46640 329.9560 0.0010 0.9010 0.0071
Flatten 1254 51.9750 0.0080 0.1120 0.0414
Activation 5016 228.8580 0.0030 0.3430 0.0456
SetValueOp 1094 7.0070 0.0050 0.0180 0.0064
pick 1094 8.5450 0.0060 0.0280 0.0078
Pooling 2508 3478.9460 0.0640 3.1720 1.3871
Convolution 2508 14199.6133 0.2260 13.2050 5.6617
CopyCPU2CPU 1254 7.1960 0.0030 0.0360 0.0057
log_softmax 1094 33.9160 0.0150 0.0550 0.0310
_backward_copy 1094 56.3520 0.0220 0.0760 0.0515
_backward_Convolution 2188 20191.5156 2.6370 12.5610 9.2283
_mul_scalar 1094 3.8730 0.0020 0.0230 0.0035
_backward_log_softmax 1094 18.4290 0.0090 0.0400 0.0168
mean 3442 65.7570 0.0080 0.0730 0.0191
_backward_mean 1094 11.9170 0.0080 0.1030 0.0109
_backward_FullyConnected 3282 619.1170 0.0280 0.5940 0.1886
WaitForVar 2348 20.8680 0.0060 0.0280 0.0089
broadcast_equal 1254 6.2320 0.0030 0.0110 0.0050
sgd_update 10940 107.0290 0.0020 0.0790 0.0098
_backward_Activation 4376 322.9420 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 42952 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3768 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 547 2167.3091 0.3060 73.5850 3.9622
MXNDArrayCreateEx 628 21.4720 0.0310 0.0640 0.0342
MXNDArrayWaitAll 548 11858.9639 5.6130 24.8060 21.6404
MXNDArraySyncCopyToCPU 1176 9252.1445 0.1180 24.1980 7.8675
MXNDArrayGetDType 1176 0.4160 0.0000 0.0010 0.0004
MXNDArrayGetShape 2980 1.3650 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1256 11.0100 0.0070 0.0280 0.0088
MXNDArrayFree 14044 143.4390 0.0000 0.9140 0.0102
MXNet C API Calls 63036 550.3930 487.3580 550.3930 31.5175
MXNet C API Concurrency 126071 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1096 0.5190 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5480 2.8510 0.0000 0.0230 0.0005
MXNDArraySetGradState 5480 2.7870 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1096 0.4670 0.0000 0.0010 0.0004
MXAutogradBackwardEx 548 297.9110 0.5250 0.6810 0.5436
MXNDArrayGetContext 8712 3.7730 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18268 464.1610 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2192 5352.4609 0.7010 3.7500 2.4418
FullyConnected 3768 361.6180 0.0080 0.3430 0.0960
argmax 1256 24.7000 0.0050 0.0480 0.0197
_backward_pick 1096 17.2250 0.0130 0.0260 0.0157
_backward_mul_scalar 1096 4.7230 0.0030 0.0160 0.0043
DeleteVariable 46720 330.5180 0.0010 0.9010 0.0071
Flatten 1256 52.0530 0.0080 0.1120 0.0414
Activation 5024 229.3900 0.0030 0.3430 0.0457
SetValueOp 1096 7.0220 0.0050 0.0180 0.0064
pick 1096 8.5630 0.0060 0.0280 0.0078
Pooling 2512 3488.3391 0.0640 3.1720 1.3887
Convolution 2512 14221.7031 0.2260 13.2050 5.6615
CopyCPU2CPU 1256 7.2080 0.0030 0.0360 0.0057
log_softmax 1096 33.9930 0.0150 0.0550 0.0310
_backward_copy 1096 56.4500 0.0220 0.0760 0.0515
_backward_Convolution 2192 20232.4336 2.6370 12.5610 9.2301
_mul_scalar 1096 3.8820 0.0020 0.0230 0.0035
_backward_log_softmax 1096 18.4750 0.0090 0.0400 0.0169
mean 3448 65.8810 0.0080 0.0730 0.0191
_backward_mean 1096 11.9420 0.0080 0.1030 0.0109
_backward_FullyConnected 3288 620.8130 0.0280 0.5940 0.1888
WaitForVar 2352 20.9090 0.0060 0.0280 0.0089
broadcast_equal 1256 6.2440 0.0030 0.0110 0.0050
sgd_update 10960 107.2640 0.0020 0.0790 0.0098
_backward_Activation 4384 323.7990 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43026 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3774 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 548 2171.2100 0.3060 73.5850 3.9621
MXNDArrayCreateEx 629 21.5060 0.0310 0.0640 0.0342
MXNDArrayWaitAll 549 11883.6494 5.6130 24.8060 21.6460
MXNDArraySyncCopyToCPU 1178 9269.9766 0.1180 24.1980 7.8693
MXNDArrayGetDType 1178 0.4160 0.0000 0.0010 0.0004
MXNDArrayGetShape 2985 1.3680 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1258 11.0280 0.0070 0.0280 0.0088
MXNDArrayFree 14067 143.6620 0.0000 0.9140 0.0102
MXNet C API Calls 63143 550.5000 487.3580 550.5000 31.5710
MXNet C API Concurrency 126285 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1098 0.5200 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5490 2.8560 0.0000 0.0230 0.0005
MXNDArraySetGradState 5490 2.7920 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1098 0.4670 0.0000 0.0010 0.0004
MXAutogradBackwardEx 549 298.4440 0.5250 0.6810 0.5436
MXNDArrayGetContext 8726 3.7780 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18299 464.9400 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2196 5364.5918 0.7010 3.7500 2.4429
FullyConnected 3774 362.5090 0.0080 0.3430 0.0961
argmax 1258 24.7490 0.0050 0.0480 0.0197
_backward_pick 1098 17.2600 0.0130 0.0260 0.0157
_backward_mul_scalar 1098 4.7310 0.0030 0.0160 0.0043
DeleteVariable 46800 331.0840 0.0010 0.9010 0.0071
Flatten 1258 52.1350 0.0080 0.1120 0.0414
Activation 5032 229.9240 0.0030 0.3430 0.0457
SetValueOp 1098 7.0360 0.0050 0.0180 0.0064
pick 1098 8.5820 0.0060 0.0280 0.0078
Pooling 2516 3497.8101 0.0640 3.1720 1.3902
Convolution 2516 14243.9004 0.2260 13.2050 5.6613
CopyCPU2CPU 1258 7.2190 0.0030 0.0360 0.0057
log_softmax 1098 34.0690 0.0150 0.0550 0.0310
_backward_copy 1098 56.5470 0.0220 0.0760 0.0515
_backward_Convolution 2196 20273.6074 2.6370 12.5610 9.2321
_mul_scalar 1098 3.8910 0.0020 0.0230 0.0035
_backward_log_softmax 1098 18.5220 0.0090 0.0400 0.0169
mean 3454 66.0080 0.0080 0.0730 0.0191
_backward_mean 1098 11.9680 0.0080 0.1030 0.0109
_backward_FullyConnected 3294 622.5060 0.0280 0.5940 0.1890
WaitForVar 2356 20.9510 0.0060 0.0280 0.0089
broadcast_equal 1258 6.2570 0.0030 0.0110 0.0050
sgd_update 10980 107.5620 0.0020 0.0790 0.0098
_backward_Activation 4392 324.6030 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43100 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3780 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 549 2175.1150 0.3060 73.5850 3.9620
MXNDArrayCreateEx 630 21.5390 0.0310 0.0640 0.0342
MXNDArrayWaitAll 550 11908.2588 5.6130 24.8060 21.6514
MXNDArraySyncCopyToCPU 1180 9287.8223 0.1180 24.1980 7.8710
MXNDArrayGetDType 1180 0.4170 0.0000 0.0010 0.0004
MXNDArrayGetShape 2990 1.3700 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1260 11.0450 0.0070 0.0280 0.0088
MXNDArrayFree 14090 143.8840 0.0000 0.9140 0.0102
MXNet C API Calls 63250 550.6070 487.3580 550.6070 31.6245
MXNet C API Concurrency 126499 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1100 0.5210 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5500 2.8590 0.0000 0.0230 0.0005
MXNDArraySetGradState 5500 2.7980 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1100 0.4690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 550 298.9740 0.5250 0.6810 0.5436
MXNDArrayGetContext 8740 3.7830 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18330 465.7140 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2200 5376.6841 0.7010 3.7500 2.4439
FullyConnected 3780 363.4000 0.0080 0.3430 0.0961
argmax 1260 24.7980 0.0050 0.0480 0.0197
_backward_pick 1100 17.2970 0.0130 0.0260 0.0157
_backward_mul_scalar 1100 4.7390 0.0030 0.0160 0.0043
DeleteVariable 46880 331.6540 0.0010 0.9010 0.0071
Flatten 1260 52.2150 0.0080 0.1120 0.0414
Activation 5040 230.4660 0.0030 0.3430 0.0457
SetValueOp 1100 7.0500 0.0050 0.0180 0.0064
pick 1100 8.6000 0.0060 0.0280 0.0078
Pooling 2520 3507.2351 0.0640 3.1720 1.3918
Convolution 2520 14266.1338 0.2260 13.2050 5.6612
CopyCPU2CPU 1260 7.2300 0.0030 0.0360 0.0057
log_softmax 1100 34.1470 0.0150 0.0550 0.0310
_backward_copy 1100 56.6430 0.0220 0.0760 0.0515
_backward_Convolution 2200 20314.6621 2.6370 12.5610 9.2339
_mul_scalar 1100 3.9000 0.0020 0.0230 0.0035
_backward_log_softmax 1100 18.5680 0.0090 0.0400 0.0169
mean 3460 66.1520 0.0080 0.0730 0.0191
_backward_mean 1100 11.9940 0.0080 0.1030 0.0109
_backward_FullyConnected 3300 624.1880 0.0280 0.5940 0.1891
WaitForVar 2360 20.9930 0.0060 0.0280 0.0089
broadcast_equal 1260 6.2690 0.0030 0.0110 0.0050
sgd_update 11000 107.8620 0.0020 0.0790 0.0098
_backward_Activation 4400 325.4060 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43174 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3786 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 550 2175.5659 0.3060 73.5850 3.9556
MXNDArrayCreateEx 631 21.5720 0.0310 0.0640 0.0342
MXNDArrayWaitAll 551 11932.7217 5.6130 24.8060 21.6565
MXNDArraySyncCopyToCPU 1182 9305.5400 0.1180 24.1980 7.8727
MXNDArrayGetDType 1182 0.4180 0.0000 0.0010 0.0004
MXNDArrayGetShape 2995 1.3720 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1262 11.0630 0.0070 0.0280 0.0088
MXNDArrayFree 14113 144.1140 0.0000 0.9140 0.0102
MXNet C API Calls 63357 550.7140 487.3580 550.7140 31.6780
MXNet C API Concurrency 126713 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1102 0.5220 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5510 2.8640 0.0000 0.0230 0.0005
MXNDArraySetGradState 5510 2.8060 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1102 0.4690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 551 299.5080 0.5250 0.6810 0.5436
MXNDArrayGetContext 8754 3.7880 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18361 466.5010 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2204 5388.9009 0.7010 3.7500 2.4451
FullyConnected 3786 364.3060 0.0080 0.3430 0.0962
argmax 1262 24.8460 0.0050 0.0480 0.0197
_backward_pick 1102 17.3320 0.0130 0.0260 0.0157
_backward_mul_scalar 1102 4.7480 0.0030 0.0160 0.0043
DeleteVariable 46960 332.2080 0.0010 0.9010 0.0071
Flatten 1262 52.2960 0.0080 0.1120 0.0414
Activation 5048 230.9980 0.0030 0.3430 0.0458
SetValueOp 1102 7.0660 0.0050 0.0180 0.0064
pick 1102 8.6190 0.0060 0.0280 0.0078
Pooling 2524 3516.7109 0.0640 3.1720 1.3933
Convolution 2524 14288.3047 0.2260 13.2050 5.6610
CopyCPU2CPU 1262 7.2410 0.0030 0.0360 0.0057
log_softmax 1102 34.2230 0.0150 0.0550 0.0311
_backward_copy 1102 56.7420 0.0220 0.0760 0.0515
_backward_Convolution 2204 20355.4141 2.6370 12.5610 9.2357
_mul_scalar 1102 3.9090 0.0020 0.0230 0.0035
_backward_log_softmax 1102 18.6140 0.0090 0.0400 0.0169
mean 3466 66.2790 0.0080 0.0730 0.0191
_backward_mean 1102 12.0210 0.0080 0.1030 0.0109
_backward_FullyConnected 3306 625.8750 0.0280 0.5940 0.1893
WaitForVar 2364 21.0340 0.0060 0.0280 0.0089
broadcast_equal 1262 6.2810 0.0030 0.0110 0.0050
sgd_update 11020 108.1550 0.0020 0.0790 0.0098
_backward_Activation 4408 326.2070 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43248 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3792 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 551 2179.4719 0.3060 73.5850 3.9555
MXNDArrayCreateEx 632 21.6060 0.0310 0.0640 0.0342
MXNDArrayWaitAll 552 11957.2861 5.6130 24.8060 21.6618
MXNDArraySyncCopyToCPU 1184 9323.2930 0.1180 24.1980 7.8744
MXNDArrayGetDType 1184 0.4190 0.0000 0.0010 0.0004
MXNDArrayGetShape 3000 1.3730 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1264 11.0810 0.0070 0.0280 0.0088
MXNDArrayFree 14136 144.3240 0.0000 0.9140 0.0102
MXNet C API Calls 63464 550.8210 487.3580 550.8210 31.7315
MXNet C API Concurrency 126927 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1104 0.5220 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5520 2.8680 0.0000 0.0230 0.0005
MXNDArraySetGradState 5520 2.8110 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1104 0.4690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 552 300.0420 0.5250 0.6810 0.5436
MXNDArrayGetContext 8768 3.7930 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18392 467.2720 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2208 5401.0488 0.7010 3.7500 2.4461
FullyConnected 3792 365.1660 0.0080 0.3430 0.0963
argmax 1264 24.8950 0.0050 0.0480 0.0197
_backward_pick 1104 17.3670 0.0130 0.0260 0.0157
_backward_mul_scalar 1104 4.7560 0.0030 0.0160 0.0043
DeleteVariable 47040 332.7450 0.0010 0.9010 0.0071
Flatten 1264 52.3800 0.0080 0.1120 0.0414
Activation 5056 231.5330 0.0030 0.3430 0.0458
SetValueOp 1104 7.0810 0.0050 0.0180 0.0064
pick 1104 8.6360 0.0060 0.0280 0.0078
Pooling 2528 3526.1101 0.0640 3.1720 1.3948
Convolution 2528 14310.4131 0.2260 13.2050 5.6608
CopyCPU2CPU 1264 7.2520 0.0030 0.0360 0.0057
log_softmax 1104 34.3000 0.0150 0.0550 0.0311
_backward_copy 1104 56.8390 0.0220 0.0760 0.0515
_backward_Convolution 2208 20396.3477 2.6370 12.5610 9.2375
_mul_scalar 1104 3.9180 0.0020 0.0230 0.0035
_backward_log_softmax 1104 18.6600 0.0090 0.0400 0.0169
mean 3472 66.4030 0.0080 0.0730 0.0191
_backward_mean 1104 12.0480 0.0080 0.1030 0.0109
_backward_FullyConnected 3312 627.5620 0.0280 0.5940 0.1895
WaitForVar 2368 21.0740 0.0060 0.0280 0.0089
broadcast_equal 1264 6.2930 0.0030 0.0110 0.0050
sgd_update 11040 108.4380 0.0020 0.0790 0.0098
_backward_Activation 4416 327.0120 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43322 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3798 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 552 2183.3840 0.3060 73.5850 3.9554
MXNDArrayCreateEx 633 21.6390 0.0310 0.0640 0.0342
MXNDArrayWaitAll 553 11981.9482 5.6130 24.8060 21.6672
MXNDArraySyncCopyToCPU 1186 9341.1475 0.1180 24.1980 7.8762
MXNDArrayGetDType 1186 0.4200 0.0000 0.0010 0.0004
MXNDArrayGetShape 3005 1.3750 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1266 11.0990 0.0070 0.0280 0.0088
MXNDArrayFree 14159 144.5450 0.0000 0.9140 0.0102
MXNet C API Calls 63571 550.9280 487.3580 550.9280 31.7850
MXNet C API Concurrency 127141 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1106 0.5240 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5530 2.8700 0.0000 0.0230 0.0005
MXNDArraySetGradState 5530 2.8150 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1106 0.4690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 553 300.5740 0.5250 0.6810 0.5435
MXNDArrayGetContext 8782 3.7990 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18423 468.0450 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2212 5413.3018 0.7010 3.7500 2.4472
FullyConnected 3798 366.0710 0.0080 0.3430 0.0964
argmax 1266 24.9440 0.0050 0.0480 0.0197
_backward_pick 1106 17.4000 0.0130 0.0260 0.0157
_backward_mul_scalar 1106 4.7650 0.0030 0.0160 0.0043
DeleteVariable 47120 333.3130 0.0010 0.9010 0.0071
Flatten 1266 52.4620 0.0080 0.1120 0.0414
Activation 5064 232.0670 0.0030 0.3430 0.0458
SetValueOp 1106 7.0950 0.0050 0.0180 0.0064
pick 1106 8.6550 0.0060 0.0280 0.0078
Pooling 2532 3535.5581 0.0640 3.1720 1.3963
Convolution 2532 14332.5654 0.2260 13.2050 5.6606
CopyCPU2CPU 1266 7.2630 0.0030 0.0360 0.0057
log_softmax 1106 34.3770 0.0150 0.0550 0.0311
_backward_copy 1106 56.9360 0.0220 0.0760 0.0515
_backward_Convolution 2212 20437.4336 2.6370 12.5610 9.2393
_mul_scalar 1106 3.9270 0.0020 0.0230 0.0036
_backward_log_softmax 1106 18.7070 0.0090 0.0400 0.0169
mean 3478 66.5280 0.0080 0.0730 0.0191
_backward_mean 1106 12.0740 0.0080 0.1030 0.0109
_backward_FullyConnected 3318 629.2580 0.0280 0.5940 0.1896
WaitForVar 2372 21.1150 0.0060 0.0280 0.0089
broadcast_equal 1266 6.3060 0.0030 0.0110 0.0050
sgd_update 11060 108.6770 0.0020 0.0790 0.0098
_backward_Activation 4424 327.8780 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43396 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3804 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 553 2187.2959 0.3060 73.5850 3.9553
MXNDArrayCreateEx 634 21.6720 0.0310 0.0640 0.0342
MXNDArrayWaitAll 554 12006.6309 5.6130 24.8060 21.6726
MXNDArraySyncCopyToCPU 1188 9358.9912 0.1180 24.1980 7.8779
MXNDArrayGetDType 1188 0.4220 0.0000 0.0010 0.0004
MXNDArrayGetShape 3010 1.3780 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1268 11.1160 0.0070 0.0280 0.0088
MXNDArrayFree 14182 144.7710 0.0000 0.9140 0.0102
MXNet C API Calls 63678 551.0350 487.3580 551.0350 31.8385
MXNet C API Concurrency 127355 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1108 0.5250 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5540 2.8750 0.0000 0.0230 0.0005
MXNDArraySetGradState 5540 2.8210 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1108 0.4690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 554 301.1050 0.5250 0.6810 0.5435
MXNDArrayGetContext 8796 3.8030 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18454 468.8150 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2216 5425.4521 0.7010 3.7500 2.4483
FullyConnected 3804 366.9630 0.0080 0.3430 0.0965
argmax 1268 24.9930 0.0050 0.0480 0.0197
_backward_pick 1108 17.4360 0.0130 0.0260 0.0157
_backward_mul_scalar 1108 4.7730 0.0030 0.0160 0.0043
DeleteVariable 47200 333.8740 0.0010 0.9010 0.0071
Flatten 1268 52.5410 0.0080 0.1120 0.0414
Activation 5072 232.5970 0.0030 0.3430 0.0459
SetValueOp 1108 7.1090 0.0050 0.0180 0.0064
pick 1108 8.6740 0.0060 0.0280 0.0078
Pooling 2536 3544.9451 0.0640 3.1720 1.3978
Convolution 2536 14354.8213 0.2260 13.2050 5.6604
CopyCPU2CPU 1268 7.2740 0.0030 0.0360 0.0057
log_softmax 1108 34.4520 0.0150 0.0550 0.0311
_backward_copy 1108 57.0320 0.0220 0.0760 0.0515
_backward_Convolution 2216 20478.5938 2.6370 12.5610 9.2412
_mul_scalar 1108 3.9350 0.0020 0.0230 0.0036
_backward_log_softmax 1108 18.7540 0.0090 0.0400 0.0169
mean 3484 66.6750 0.0080 0.0730 0.0191
_backward_mean 1108 12.1000 0.0080 0.1030 0.0109
_backward_FullyConnected 3324 630.9370 0.0280 0.5940 0.1898
WaitForVar 2376 21.1560 0.0060 0.0280 0.0089
broadcast_equal 1268 6.3180 0.0030 0.0110 0.0050
sgd_update 11080 108.9750 0.0020 0.0790 0.0098
_backward_Activation 4432 328.6770 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43470 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3810 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 554 2191.2041 0.3060 73.5850 3.9552
MXNDArrayCreateEx 635 21.7050 0.0310 0.0640 0.0342
MXNDArrayWaitAll 555 12031.1885 5.6130 24.8060 21.6778
MXNDArraySyncCopyToCPU 1190 9376.8252 0.1180 24.1980 7.8797
MXNDArrayGetDType 1190 0.4230 0.0000 0.0010 0.0004
MXNDArrayGetShape 3015 1.3800 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1270 11.1330 0.0070 0.0280 0.0088
MXNDArrayFree 14205 144.9960 0.0000 0.9140 0.0102
MXNet C API Calls 63785 551.1420 487.3580 551.1420 31.8920
MXNet C API Concurrency 127569 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1110 0.5260 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5550 2.8830 0.0000 0.0230 0.0005
MXNDArraySetGradState 5550 2.8250 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1110 0.4700 0.0000 0.0010 0.0004
MXAutogradBackwardEx 555 301.6330 0.5250 0.6810 0.5435
MXNDArrayGetContext 8810 3.8090 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18485 469.5930 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2220 5437.5000 0.7010 3.7500 2.4493
FullyConnected 3810 367.8670 0.0080 0.3430 0.0966
argmax 1270 25.0420 0.0050 0.0480 0.0197
_backward_pick 1110 17.4700 0.0130 0.0260 0.0157
_backward_mul_scalar 1110 4.7800 0.0030 0.0160 0.0043
DeleteVariable 47280 334.4300 0.0010 0.9010 0.0071
Flatten 1270 52.6210 0.0080 0.1120 0.0414
Activation 5080 233.1300 0.0030 0.3430 0.0459
SetValueOp 1110 7.1230 0.0050 0.0180 0.0064
pick 1110 8.6930 0.0060 0.0280 0.0078
Pooling 2540 3554.3950 0.0640 3.1720 1.3994
Convolution 2540 14377.0020 0.2260 13.2050 5.6602
CopyCPU2CPU 1270 7.2850 0.0030 0.0360 0.0057
log_softmax 1110 34.5340 0.0150 0.0550 0.0311
_backward_copy 1110 57.1280 0.0220 0.0760 0.0515
_backward_Convolution 2220 20519.5527 2.6370 12.5610 9.2430
_mul_scalar 1110 3.9430 0.0020 0.0230 0.0036
_backward_log_softmax 1110 18.8020 0.0090 0.0400 0.0169
mean 3490 66.8010 0.0080 0.0730 0.0191
_backward_mean 1110 12.1260 0.0080 0.1030 0.0109
_backward_FullyConnected 3330 632.6310 0.0280 0.5940 0.1900
WaitForVar 2380 21.1970 0.0060 0.0280 0.0089
broadcast_equal 1270 6.3300 0.0030 0.0110 0.0050
sgd_update 11100 109.2130 0.0020 0.0790 0.0098
_backward_Activation 4440 329.5400 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43544 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3816 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 555 2195.0979 0.3060 73.5850 3.9551
MXNDArrayCreateEx 636 21.7390 0.0310 0.0640 0.0342
MXNDArrayWaitAll 556 12055.6611 5.6130 24.8060 21.6828
MXNDArraySyncCopyToCPU 1192 9394.6514 0.1180 24.1980 7.8814
MXNDArrayGetDType 1192 0.4240 0.0000 0.0010 0.0004
MXNDArrayGetShape 3020 1.3820 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1272 11.1510 0.0070 0.0280 0.0088
MXNDArrayFree 14228 145.2120 0.0000 0.9140 0.0102
MXNet C API Calls 63892 551.2490 487.3580 551.2490 31.9455
MXNet C API Concurrency 127783 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1112 0.5280 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5560 2.8900 0.0000 0.0230 0.0005
MXNDArraySetGradState 5560 2.8310 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1112 0.4710 0.0000 0.0010 0.0004
MXAutogradBackwardEx 556 302.1630 0.5250 0.6810 0.5435
MXNDArrayGetContext 8824 3.8150 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18516 470.3660 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2224 5449.6699 0.7010 3.7500 2.4504
FullyConnected 3816 368.7680 0.0080 0.3430 0.0966
argmax 1272 25.0930 0.0050 0.0480 0.0197
_backward_pick 1112 17.5040 0.0130 0.0260 0.0157
_backward_mul_scalar 1112 4.7880 0.0030 0.0160 0.0043
DeleteVariable 47360 334.9640 0.0010 0.9010 0.0071
Flatten 1272 52.7010 0.0080 0.1120 0.0414
Activation 5088 233.6650 0.0030 0.3430 0.0459
SetValueOp 1112 7.1380 0.0050 0.0180 0.0064
pick 1112 8.7120 0.0060 0.0280 0.0078
Pooling 2544 3563.8311 0.0640 3.1720 1.4009
Convolution 2544 14399.1016 0.2260 13.2050 5.6600
CopyCPU2CPU 1272 7.2960 0.0030 0.0360 0.0057
log_softmax 1112 34.6100 0.0150 0.0550 0.0311
_backward_copy 1112 57.2240 0.0220 0.0760 0.0515
_backward_Convolution 2224 20560.3242 2.6370 12.5610 9.2448
_mul_scalar 1112 3.9520 0.0020 0.0230 0.0036
_backward_log_softmax 1112 18.8490 0.0090 0.0400 0.0170
mean 3496 66.9280 0.0080 0.0730 0.0191
_backward_mean 1112 12.1530 0.0080 0.1030 0.0109
_backward_FullyConnected 3336 634.3270 0.0280 0.5940 0.1901
WaitForVar 2384 21.2360 0.0060 0.0280 0.0089
broadcast_equal 1272 6.3420 0.0030 0.0110 0.0050
sgd_update 11120 109.5110 0.0020 0.0790 0.0098
_backward_Activation 4448 330.3400 0.0070 0.3900 0.0743
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43618 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3822 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 556 2199.0071 0.3060 73.5850 3.9550
MXNDArrayCreateEx 637 21.7720 0.0310 0.0640 0.0342
MXNDArrayWaitAll 557 12077.4805 5.6130 24.8060 21.6831
MXNDArraySyncCopyToCPU 1194 9410.6953 0.1180 24.1980 7.8817
MXNDArrayGetDType 1194 0.4250 0.0000 0.0010 0.0004
MXNDArrayGetShape 3025 1.3850 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1274 11.1690 0.0070 0.0280 0.0088
MXNDArrayFree 14251 145.4550 0.0000 0.9140 0.0102
MXNet C API Calls 63999 551.3560 487.3580 551.3560 31.9990
MXNet C API Concurrency 127997 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1114 0.5280 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5570 2.8980 0.0000 0.0230 0.0005
MXNDArraySetGradState 5570 2.8360 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1114 0.4710 0.0000 0.0010 0.0004
MXAutogradBackwardEx 557 302.7180 0.5250 0.6810 0.5435
MXNDArrayGetContext 8838 3.8190 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18547 471.1640 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2228 5459.3501 0.7010 3.7500 2.4503
FullyConnected 3822 369.3190 0.0080 0.3430 0.0966
argmax 1274 25.1310 0.0050 0.0480 0.0197
_backward_pick 1114 17.5370 0.0130 0.0260 0.0157
_backward_mul_scalar 1114 4.7970 0.0030 0.0160 0.0043
DeleteVariable 47440 335.5600 0.0010 0.9010 0.0071
Flatten 1274 52.7990 0.0080 0.1120 0.0414
Activation 5096 237.6890 0.0030 1.9440 0.0466
SetValueOp 1114 7.1520 0.0050 0.0180 0.0064
pick 1114 8.7270 0.0060 0.0280 0.0078
Pooling 2548 3569.0830 0.0640 3.1720 1.4007
Convolution 2548 14421.2881 0.2260 13.2050 5.6598
CopyCPU2CPU 1274 7.3060 0.0030 0.0360 0.0057
log_softmax 1114 34.6700 0.0150 0.0550 0.0311
_backward_copy 1114 57.3260 0.0220 0.0760 0.0515
_backward_Convolution 2228 20597.3320 2.6370 12.5610 9.2448
_mul_scalar 1114 3.9590 0.0020 0.0230 0.0036
_backward_log_softmax 1114 18.8800 0.0090 0.0400 0.0169
mean 3502 67.0450 0.0080 0.0730 0.0191
_backward_mean 1114 12.1800 0.0080 0.1030 0.0109
_backward_FullyConnected 3342 635.3970 0.0280 0.5940 0.1901
WaitForVar 2388 21.2730 0.0060 0.0280 0.0089
broadcast_equal 1274 6.3520 0.0030 0.0110 0.0050
sgd_update 11140 109.6600 0.0020 0.0790 0.0098
_backward_Activation 4456 330.9330 0.0070 0.3900 0.0743
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43692 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3828 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 557 2202.9021 0.3060 73.5850 3.9549
MXNDArrayCreateEx 638 21.8060 0.0310 0.0640 0.0342
MXNDArrayWaitAll 558 12102.4678 5.6130 24.9880 21.6890
MXNDArraySyncCopyToCPU 1196 9436.8516 0.1180 24.1980 7.8903
MXNDArrayGetDType 1196 0.4250 0.0000 0.0010 0.0004
MXNDArrayGetShape 3030 1.3900 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1276 11.1860 0.0070 0.0280 0.0088
MXNDArrayFree 14274 145.6800 0.0000 0.9140 0.0102
MXNet C API Calls 64106 551.4630 487.3580 551.4630 32.0525
MXNet C API Concurrency 128211 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1116 0.5300 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5580 2.9000 0.0000 0.0230 0.0005
MXNDArraySetGradState 5580 2.8390 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1116 0.4720 0.0000 0.0010 0.0004
MXAutogradBackwardEx 558 303.2540 0.5250 0.6810 0.5435
MXNDArrayGetContext 8852 3.8230 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18578 471.9750 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2232 5470.3408 0.7010 3.7500 2.4509
FullyConnected 3828 370.2100 0.0080 0.3430 0.0967
argmax 1276 25.1840 0.0050 0.0480 0.0197
_backward_pick 1116 17.5780 0.0130 0.0260 0.0158
_backward_mul_scalar 1116 4.8120 0.0030 0.0160 0.0043
DeleteVariable 47520 336.1320 0.0010 0.9010 0.0071
Flatten 1276 52.8920 0.0080 0.1120 0.0415
Activation 5104 238.1110 0.0030 1.9440 0.0467
SetValueOp 1116 7.1700 0.0050 0.0180 0.0064
pick 1116 8.7500 0.0060 0.0280 0.0078
Pooling 2552 3578.6089 0.0640 3.1720 1.4023
Convolution 2552 14459.6533 0.2260 13.2050 5.6660
CopyCPU2CPU 1276 7.3180 0.0030 0.0360 0.0057
log_softmax 1116 34.7490 0.0150 0.0550 0.0311
_backward_copy 1116 57.4450 0.0220 0.0760 0.0515
_backward_Convolution 2232 20640.6133 2.6370 12.5610 9.2476
_mul_scalar 1116 3.9700 0.0020 0.0230 0.0036
_backward_log_softmax 1116 18.9260 0.0090 0.0400 0.0170
mean 3508 67.2030 0.0080 0.0730 0.0192
_backward_mean 1116 12.2100 0.0080 0.1030 0.0109
_backward_FullyConnected 3348 637.1240 0.0280 0.5940 0.1903
WaitForVar 2392 21.3230 0.0060 0.0280 0.0089
broadcast_equal 1276 6.3660 0.0030 0.0110 0.0050
sgd_update 11160 109.9010 0.0020 0.0790 0.0098
_backward_Activation 4464 331.5860 0.0070 0.3900 0.0743
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43766 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3834 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 558 2206.7971 0.3060 73.5850 3.9548
MXNDArrayCreateEx 639 21.8390 0.0310 0.0640 0.0342
MXNDArrayWaitAll 559 12124.1270 5.6130 24.9880 21.6890
MXNDArraySyncCopyToCPU 1198 9451.2080 0.1180 24.1980 7.8892
MXNDArrayGetDType 1198 0.4260 0.0000 0.0010 0.0004
MXNDArrayGetShape 3035 1.3930 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1278 11.2050 0.0070 0.0280 0.0088
MXNDArrayFree 14297 145.9250 0.0000 0.9140 0.0102
MXNet C API Calls 64213 551.5700 487.3580 551.5700 32.1060
MXNet C API Concurrency 128425 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1118 0.5320 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5590 2.9050 0.0000 0.0230 0.0005
MXNDArraySetGradState 5590 2.8450 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1118 0.4720 0.0000 0.0010 0.0004
MXAutogradBackwardEx 559 303.7850 0.5250 0.6810 0.5434
MXNDArrayGetContext 8866 3.8290 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18609 472.7440 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2236 5479.9468 0.7010 3.7500 2.4508
FullyConnected 3834 370.7540 0.0080 0.3430 0.0967
argmax 1278 25.2220 0.0050 0.0480 0.0197
_backward_pick 1118 17.6110 0.0130 0.0260 0.0158
_backward_mul_scalar 1118 4.8200 0.0030 0.0160 0.0043
DeleteVariable 47600 336.7110 0.0010 0.9010 0.0071
Flatten 1278 52.9700 0.0080 0.1120 0.0414
Activation 5112 238.4620 0.0030 1.9440 0.0466
SetValueOp 1118 7.1830 0.0050 0.0180 0.0064
pick 1118 8.7650 0.0060 0.0280 0.0078
Pooling 2556 3583.9341 0.0640 3.1720 1.4022
Convolution 2556 14481.8242 0.2260 13.2050 5.6658
CopyCPU2CPU 1278 7.3270 0.0030 0.0360 0.0057
log_softmax 1118 34.8090 0.0150 0.0550 0.0311
_backward_copy 1118 57.5460 0.0220 0.0760 0.0515
_backward_Convolution 2236 20677.3418 2.6370 12.5610 9.2475
_mul_scalar 1118 3.9770 0.0020 0.0230 0.0036
_backward_log_softmax 1118 18.9590 0.0090 0.0400 0.0170
mean 3514 67.3220 0.0080 0.0730 0.0192
_backward_mean 1118 12.2320 0.0080 0.1030 0.0109
_backward_FullyConnected 3354 638.1920 0.0280 0.5940 0.1903
WaitForVar 2396 21.3600 0.0060 0.0280 0.0089
broadcast_equal 1278 6.3760 0.0030 0.0110 0.0050
sgd_update 11180 110.1030 0.0020 0.0790 0.0098
_backward_Activation 4472 332.1550 0.0070 0.3900 0.0743
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43840 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3840 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 559 2210.7080 0.3060 73.5850 3.9548
MXNDArrayCreateEx 640 21.8720 0.0310 0.0640 0.0342
MXNDArrayWaitAll 560 12145.8145 5.6130 24.9880 21.6890
MXNDArraySyncCopyToCPU 1200 9465.4893 0.1180 24.1980 7.8879
MXNDArrayGetDType 1200 0.4270 0.0000 0.0010 0.0004
MXNDArrayGetShape 3040 1.3950 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1280 11.2220 0.0070 0.0280 0.0088
MXNDArrayFree 14320 146.1470 0.0000 0.9140 0.0102
MXNet C API Calls 64320 551.6770 487.3580 551.6770 32.1595
MXNet C API Concurrency 128639 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1120 0.5320 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5600 2.9090 0.0000 0.0230 0.0005
MXNDArraySetGradState 5600 2.8510 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1120 0.4740 0.0000 0.0010 0.0004
MXAutogradBackwardEx 560 304.3380 0.5250 0.6810 0.5435
MXNDArrayGetContext 8880 3.8380 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18640 473.5260 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2240 5489.6152 0.7010 3.7500 2.4507
FullyConnected 3840 371.2740 0.0080 0.3430 0.0967
argmax 1280 25.2590 0.0050 0.0480 0.0197
_backward_pick 1120 17.6430 0.0130 0.0260 0.0158
_backward_mul_scalar 1120 4.8280 0.0030 0.0160 0.0043
DeleteVariable 47680 337.2180 0.0010 0.9010 0.0071
Flatten 1280 53.0470 0.0080 0.1120 0.0414
Activation 5120 238.8140 0.0030 1.9440 0.0466
SetValueOp 1120 7.1950 0.0050 0.0180 0.0064
pick 1120 8.7800 0.0060 0.0280 0.0078
Pooling 2560 3589.3301 0.0640 3.1720 1.4021
Convolution 2560 14503.8926 0.2260 13.2050 5.6656
CopyCPU2CPU 1280 7.3360 0.0030 0.0360 0.0057
log_softmax 1120 34.8690 0.0150 0.0550 0.0311
_backward_copy 1120 57.6500 0.0220 0.0760 0.0515
_backward_Convolution 2240 20714.0645 2.6370 12.5610 9.2473
_mul_scalar 1120 3.9840 0.0020 0.0230 0.0036
_backward_log_softmax 1120 18.9910 0.0090 0.0400 0.0170
mean 3520 67.4330 0.0080 0.0730 0.0192
_backward_mean 1120 12.2530 0.0080 0.1030 0.0109
_backward_FullyConnected 3360 639.2530 0.0280 0.5940 0.1903
WaitForVar 2400 21.3940 0.0060 0.0280 0.0089
broadcast_equal 1280 6.3860 0.0030 0.0110 0.0050
sgd_update 11200 110.3040 0.0020 0.0790 0.0098
_backward_Activation 4480 332.7040 0.0070 0.3900 0.0743
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43914 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3846 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 560 2214.6060 0.3060 73.5850 3.9547
MXNDArrayCreateEx 641 21.9060 0.0310 0.0640 0.0342
MXNDArrayWaitAll 561 12167.6533 5.6130 24.9880 21.6892
MXNDArraySyncCopyToCPU 1202 9479.8008 0.1180 24.1980 7.8867
MXNDArrayGetDType 1202 0.4270 0.0000 0.0010 0.0004
MXNDArrayGetShape 3045 1.3980 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1282 11.2390 0.0070 0.0280 0.0088
MXNDArrayFree 14343 146.3900 0.0000 0.9140 0.0102
MXNet C API Calls 64427 551.7840 487.3580 551.7840 32.2130
MXNet C API Concurrency 128853 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1122 0.5340 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5610 2.9120 0.0000 0.0230 0.0005
MXNDArraySetGradState 5610 2.8570 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1122 0.4760 0.0000 0.0010 0.0004
MXAutogradBackwardEx 561 304.8750 0.5250 0.6810 0.5434
MXNDArrayGetContext 8894 3.8440 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18671 474.3280 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2244 5499.3101 0.7010 3.7500 2.4507
FullyConnected 3846 371.7980 0.0080 0.3430 0.0967
argmax 1282 25.2990 0.0050 0.0480 0.0197
_backward_pick 1122 17.6750 0.0130 0.0260 0.0158
_backward_mul_scalar 1122 4.8370 0.0030 0.0160 0.0043
DeleteVariable 47760 337.7880 0.0010 0.9010 0.0071
Flatten 1282 53.1270 0.0080 0.1120 0.0414
Activation 5128 239.1660 0.0030 1.9440 0.0466
SetValueOp 1122 7.2080 0.0050 0.0180 0.0064
pick 1122 8.7950 0.0060 0.0280 0.0078
Pooling 2564 3594.7061 0.0640 3.1720 1.4020
Convolution 2564 14526.0557 0.2260 13.2050 5.6654
CopyCPU2CPU 1282 7.3450 0.0030 0.0360 0.0057
log_softmax 1122 34.9290 0.0150 0.0550 0.0311
_backward_copy 1122 57.7510 0.0220 0.0760 0.0515
_backward_Convolution 2244 20751.1172 2.6370 12.5610 9.2474
_mul_scalar 1122 3.9920 0.0020 0.0230 0.0036
_backward_log_softmax 1122 19.0230 0.0090 0.0400 0.0170
mean 3526 67.5530 0.0080 0.0730 0.0192
_backward_mean 1122 12.2740 0.0080 0.1030 0.0109
_backward_FullyConnected 3366 640.3210 0.0280 0.5940 0.1902
WaitForVar 2404 21.4300 0.0060 0.0280 0.0089
broadcast_equal 1282 6.3950 0.0030 0.0110 0.0050
sgd_update 11220 110.5050 0.0020 0.0790 0.0098
_backward_Activation 4488 333.2870 0.0070 0.3900 0.0743
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 43988 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3852 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 561 2218.5161 0.3060 73.5850 3.9546
MXNDArrayCreateEx 642 21.9400 0.0310 0.0640 0.0342
MXNDArrayWaitAll 562 12192.5322 5.6130 24.9880 21.6949
MXNDArraySyncCopyToCPU 1204 9505.8604 0.1180 24.1980 7.8952
MXNDArrayGetDType 1204 0.4270 0.0000 0.0010 0.0004
MXNDArrayGetShape 3050 1.4000 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1284 11.2570 0.0070 0.0280 0.0088
MXNDArrayFree 14366 146.6230 0.0000 0.9140 0.0102
MXNet C API Calls 64534 551.8910 487.3580 551.8910 32.2665
MXNet C API Concurrency 129067 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1124 0.5360 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5620 2.9220 0.0000 0.0230 0.0005
MXNDArraySetGradState 5620 2.8630 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1124 0.4770 0.0000 0.0010 0.0004
MXAutogradBackwardEx 562 305.4040 0.5250 0.6810 0.5434
MXNDArrayGetContext 8908 3.8500 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18702 475.1120 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2248 5510.2368 0.7010 3.7500 2.4512
FullyConnected 3852 372.6730 0.0080 0.3430 0.0967
argmax 1284 25.3520 0.0050 0.0480 0.0197
_backward_pick 1124 17.7140 0.0130 0.0260 0.0158
_backward_mul_scalar 1124 4.8520 0.0030 0.0160 0.0043
DeleteVariable 47840 338.3810 0.0010 0.9010 0.0071
Flatten 1284 53.2200 0.0080 0.1120 0.0414
Activation 5136 239.5940 0.0030 1.9440 0.0466
SetValueOp 1124 7.2250 0.0050 0.0180 0.0064
pick 1124 8.8170 0.0060 0.0280 0.0078
Pooling 2568 3604.1509 0.0640 3.1720 1.4035
Convolution 2568 14564.3945 0.2260 13.2050 5.6715
CopyCPU2CPU 1284 7.3570 0.0030 0.0360 0.0057
log_softmax 1124 35.0080 0.0150 0.0550 0.0311
_backward_copy 1124 57.8690 0.0220 0.0760 0.0515
_backward_Convolution 2248 20794.2266 2.6370 12.5610 9.2501
_mul_scalar 1124 4.0040 0.0020 0.0230 0.0036
_backward_log_softmax 1124 19.0720 0.0090 0.0400 0.0170
mean 3532 67.7120 0.0080 0.0730 0.0192
_backward_mean 1124 12.3040 0.0080 0.1030 0.0109
_backward_FullyConnected 3372 642.0270 0.0280 0.5940 0.1904
WaitForVar 2408 21.4740 0.0060 0.0280 0.0089
broadcast_equal 1284 6.4110 0.0030 0.0110 0.0050
sgd_update 11240 110.8070 0.0020 0.0790 0.0099
_backward_Activation 4496 333.8920 0.0070 0.3900 0.0743
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44062 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3858 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 562 2222.4089 0.3060 73.5850 3.9545
MXNDArrayCreateEx 643 21.9740 0.0310 0.0640 0.0342
MXNDArrayWaitAll 563 12214.2861 5.6130 24.9880 21.6950
MXNDArraySyncCopyToCPU 1206 9520.2012 0.1180 24.1980 7.8940
MXNDArrayGetDType 1206 0.4290 0.0000 0.0010 0.0004
MXNDArrayGetShape 3055 1.4030 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1286 11.2750 0.0070 0.0280 0.0088
MXNDArrayFree 14389 146.8530 0.0000 0.9140 0.0102
MXNet C API Calls 64641 551.9980 487.3580 551.9980 32.3200
MXNet C API Concurrency 129281 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1126 0.5360 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5630 2.9260 0.0000 0.0230 0.0005
MXNDArraySetGradState 5630 2.8660 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1126 0.4780 0.0000 0.0010 0.0004
MXAutogradBackwardEx 563 305.9340 0.5250 0.6810 0.5434
MXNDArrayGetContext 8922 3.8540 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18733 475.8960 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2252 5520.0010 0.7010 3.7500 2.4512
FullyConnected 3858 373.2510 0.0080 0.3430 0.0967
argmax 1286 25.3920 0.0050 0.0480 0.0197
_backward_pick 1126 17.7460 0.0130 0.0260 0.0158
_backward_mul_scalar 1126 4.8610 0.0030 0.0160 0.0043
DeleteVariable 47920 338.9080 0.0010 0.9010 0.0071
Flatten 1286 53.2980 0.0080 0.1120 0.0414
Activation 5144 239.9420 0.0030 1.9440 0.0466
SetValueOp 1126 7.2380 0.0050 0.0180 0.0064
pick 1126 8.8320 0.0060 0.0280 0.0078
Pooling 2572 3609.5100 0.0640 3.1720 1.4034
Convolution 2572 14586.5830 0.2260 13.2050 5.6713
CopyCPU2CPU 1286 7.3670 0.0030 0.0360 0.0057
log_softmax 1126 35.0680 0.0150 0.0550 0.0311
_backward_copy 1126 57.9730 0.0220 0.0760 0.0515
_backward_Convolution 2252 20830.9883 2.6370 12.5610 9.2500
_mul_scalar 1126 4.0120 0.0020 0.0230 0.0036
_backward_log_softmax 1126 19.1040 0.0090 0.0400 0.0170
mean 3538 67.8300 0.0080 0.0730 0.0192
_backward_mean 1126 12.3260 0.0080 0.1030 0.0109
_backward_FullyConnected 3378 643.0940 0.0280 0.5940 0.1904
WaitForVar 2412 21.5080 0.0060 0.0280 0.0089
broadcast_equal 1286 6.4200 0.0030 0.0110 0.0050
sgd_update 11260 111.0060 0.0020 0.0790 0.0099
_backward_Activation 4504 334.4400 0.0070 0.3900 0.0743
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44136 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3864 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 563 2226.3101 0.3060 73.5850 3.9544
MXNDArrayCreateEx 644 22.0080 0.0310 0.0640 0.0342
MXNDArrayWaitAll 564 12235.9248 5.6130 24.9880 21.6949
MXNDArraySyncCopyToCPU 1208 9534.5576 0.1180 24.1980 7.8928
MXNDArrayGetDType 1208 0.4290 0.0000 0.0010 0.0004
MXNDArrayGetShape 3060 1.4060 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1288 11.2920 0.0070 0.0280 0.0088
MXNDArrayFree 14412 147.0770 0.0000 0.9140 0.0102
MXNet C API Calls 64748 552.1050 487.3580 552.1050 32.3735
MXNet C API Concurrency 129495 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1128 0.5370 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5640 2.9300 0.0000 0.0230 0.0005
MXNDArraySetGradState 5640 2.8660 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1128 0.4800 0.0000 0.0010 0.0004
MXAutogradBackwardEx 564 306.4660 0.5250 0.6810 0.5434
MXNDArrayGetContext 8936 3.8620 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18764 476.6750 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2256 5529.6528 0.7010 3.7500 2.4511
FullyConnected 3864 373.7720 0.0080 0.3430 0.0967
argmax 1288 25.4300 0.0050 0.0480 0.0197
_backward_pick 1128 17.7770 0.0130 0.0260 0.0158
_backward_mul_scalar 1128 4.8690 0.0030 0.0160 0.0043
DeleteVariable 48000 339.4390 0.0010 0.9010 0.0071
Flatten 1288 53.3770 0.0080 0.1120 0.0414
Activation 5152 240.2950 0.0030 1.9440 0.0466
SetValueOp 1128 7.2500 0.0050 0.0180 0.0064
pick 1128 8.8470 0.0060 0.0280 0.0078
Pooling 2576 3614.9060 0.0640 3.1720 1.4033
Convolution 2576 14608.6934 0.2260 13.2050 5.6711
CopyCPU2CPU 1288 7.3770 0.0030 0.0360 0.0057
log_softmax 1128 35.1270 0.0150 0.0550 0.0311
_backward_copy 1128 58.0750 0.0220 0.0760 0.0515
_backward_Convolution 2256 20867.7168 2.6370 12.5610 9.2499
_mul_scalar 1128 4.0190 0.0020 0.0230 0.0036
_backward_log_softmax 1128 19.1350 0.0090 0.0400 0.0170
mean 3544 67.9440 0.0080 0.0730 0.0192
_backward_mean 1128 12.3470 0.0080 0.1030 0.0109
_backward_FullyConnected 3384 644.1580 0.0280 0.5940 0.1904
WaitForVar 2416 21.5420 0.0060 0.0280 0.0089
broadcast_equal 1288 6.4300 0.0030 0.0110 0.0050
sgd_update 11280 111.2070 0.0020 0.0790 0.0099
_backward_Activation 4512 334.9870 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44210 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3870 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 564 2230.2090 0.3060 73.5850 3.9543
MXNDArrayCreateEx 645 22.0410 0.0310 0.0640 0.0342
MXNDArrayWaitAll 565 12257.7529 5.6130 24.9880 21.6951
MXNDArraySyncCopyToCPU 1210 9548.9150 0.1180 24.1980 7.8917
MXNDArrayGetDType 1210 0.4290 0.0000 0.0010 0.0004
MXNDArrayGetShape 3065 1.4070 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1290 11.3090 0.0070 0.0280 0.0088
MXNDArrayFree 14435 147.3000 0.0000 0.9140 0.0102
MXNet C API Calls 64855 552.2120 487.3580 552.2120 32.4270
MXNet C API Concurrency 129709 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1130 0.5370 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5650 2.9340 0.0000 0.0230 0.0005
MXNDArraySetGradState 5650 2.8700 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1130 0.4800 0.0000 0.0010 0.0004
MXAutogradBackwardEx 565 307.0010 0.5250 0.6810 0.5434
MXNDArrayGetContext 8950 3.8710 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18795 477.4460 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2260 5539.3379 0.7010 3.7500 2.4510
FullyConnected 3870 374.3160 0.0080 0.3430 0.0967
argmax 1290 25.4680 0.0050 0.0480 0.0197
_backward_pick 1130 17.8080 0.0130 0.0260 0.0158
_backward_mul_scalar 1130 4.8780 0.0030 0.0160 0.0043
DeleteVariable 48080 339.9730 0.0010 0.9010 0.0071
Flatten 1290 53.4540 0.0080 0.1120 0.0414
Activation 5160 240.6470 0.0030 1.9440 0.0466
SetValueOp 1130 7.2620 0.0050 0.0180 0.0064
pick 1130 8.8620 0.0060 0.0280 0.0078
Pooling 2580 3620.2329 0.0640 3.1720 1.4032
Convolution 2580 14630.8486 0.2260 13.2050 5.6709
CopyCPU2CPU 1290 7.3870 0.0030 0.0360 0.0057
log_softmax 1130 35.1860 0.0150 0.0550 0.0311
_backward_copy 1130 58.1770 0.0220 0.0760 0.0515
_backward_Convolution 2260 20904.7070 2.6370 12.5610 9.2499
_mul_scalar 1130 4.0260 0.0020 0.0230 0.0036
_backward_log_softmax 1130 19.1670 0.0090 0.0400 0.0170
mean 3550 68.0580 0.0080 0.0730 0.0192
_backward_mean 1130 12.3680 0.0080 0.1030 0.0109
_backward_FullyConnected 3390 645.2210 0.0280 0.5940 0.1903
WaitForVar 2420 21.5780 0.0060 0.0280 0.0089
broadcast_equal 1290 6.4390 0.0030 0.0110 0.0050
sgd_update 11300 111.4260 0.0020 0.0790 0.0099
_backward_Activation 4520 335.5460 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44284 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3876 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 565 2234.0840 0.3060 73.5850 3.9541
MXNDArrayCreateEx 646 22.0750 0.0310 0.0640 0.0342
MXNDArrayWaitAll 566 12282.7256 5.6130 24.9880 21.7009
MXNDArraySyncCopyToCPU 1212 9574.6533 0.1180 24.1980 7.8999
MXNDArrayGetDType 1212 0.4310 0.0000 0.0010 0.0004
MXNDArrayGetShape 3070 1.4110 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1292 11.3270 0.0070 0.0280 0.0088
MXNDArrayFree 14458 147.5300 0.0000 0.9140 0.0102
MXNet C API Calls 64962 552.3190 487.3580 552.3190 32.4805
MXNet C API Concurrency 129923 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1132 0.5380 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5660 2.9400 0.0000 0.0230 0.0005
MXNDArraySetGradState 5660 2.8740 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1132 0.4810 0.0000 0.0010 0.0004
MXAutogradBackwardEx 566 307.5590 0.5250 0.6810 0.5434
MXNDArrayGetContext 8964 3.8760 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18826 478.2370 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2264 5550.3408 0.7010 3.7500 2.4516
FullyConnected 3876 375.2010 0.0080 0.3430 0.0968
argmax 1292 25.5230 0.0050 0.0480 0.0198
_backward_pick 1132 17.8500 0.0130 0.0260 0.0158
_backward_mul_scalar 1132 4.8940 0.0030 0.0160 0.0043
DeleteVariable 48160 340.5760 0.0010 0.9010 0.0071
Flatten 1292 53.5470 0.0080 0.1120 0.0414
Activation 5168 241.0710 0.0030 1.9440 0.0466
SetValueOp 1132 7.2810 0.0050 0.0180 0.0064
pick 1132 8.8840 0.0060 0.0280 0.0078
Pooling 2584 3629.7629 0.0640 3.1720 1.4047
Convolution 2584 14669.2891 0.2260 13.2050 5.6770
CopyCPU2CPU 1292 7.3990 0.0030 0.0360 0.0057
log_softmax 1132 35.2640 0.0150 0.0550 0.0312
_backward_copy 1132 58.2970 0.0220 0.0760 0.0515
_backward_Convolution 2264 20947.9609 2.6370 12.5610 9.2526
_mul_scalar 1132 4.0380 0.0020 0.0230 0.0036
_backward_log_softmax 1132 19.2150 0.0090 0.0400 0.0170
mean 3556 68.2160 0.0080 0.0730 0.0192
_backward_mean 1132 12.3990 0.0080 0.1030 0.0110
_backward_FullyConnected 3396 646.9680 0.0280 0.5940 0.1905
WaitForVar 2424 21.6250 0.0060 0.0280 0.0089
broadcast_equal 1292 6.4550 0.0030 0.0110 0.0050
sgd_update 11320 111.7210 0.0020 0.0790 0.0099
_backward_Activation 4528 336.1440 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44358 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3882 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 566 2238.0220 0.3060 73.5850 3.9541
MXNDArrayCreateEx 647 22.1090 0.0310 0.0640 0.0342
MXNDArrayWaitAll 567 12304.3496 5.6130 24.9880 21.7008
MXNDArraySyncCopyToCPU 1214 9589.0381 0.1180 24.1980 7.8987
MXNDArrayGetDType 1214 0.4320 0.0000 0.0010 0.0004
MXNDArrayGetShape 3075 1.4120 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1294 11.3440 0.0070 0.0280 0.0088
MXNDArrayFree 14481 147.7480 0.0000 0.9140 0.0102
MXNet C API Calls 65069 552.4260 487.3580 552.4260 32.5340
MXNet C API Concurrency 130137 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1134 0.5390 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5670 2.9450 0.0000 0.0230 0.0005
MXNDArraySetGradState 5670 2.8800 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1134 0.4820 0.0000 0.0010 0.0004
MXAutogradBackwardEx 567 308.0920 0.5250 0.6810 0.5434
MXNDArrayGetContext 8978 3.8830 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18857 479.0220 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2268 5559.9321 0.7010 3.7500 2.4515
FullyConnected 3882 375.7240 0.0080 0.3430 0.0968
argmax 1294 25.5610 0.0050 0.0480 0.0198
_backward_pick 1134 17.8810 0.0130 0.0260 0.0158
_backward_mul_scalar 1134 4.9030 0.0030 0.0160 0.0043
DeleteVariable 48240 341.1090 0.0010 0.9010 0.0071
Flatten 1294 53.6250 0.0080 0.1120 0.0414
Activation 5176 241.4230 0.0030 1.9440 0.0466
SetValueOp 1134 7.2940 0.0050 0.0180 0.0064
pick 1134 8.8990 0.0060 0.0280 0.0078
Pooling 2588 3635.1470 0.0640 3.1720 1.4046
Convolution 2588 14691.4912 0.2260 13.2050 5.6768
CopyCPU2CPU 1294 7.4090 0.0030 0.0360 0.0057
log_softmax 1134 35.3250 0.0150 0.0550 0.0312
_backward_copy 1134 58.4010 0.0220 0.0760 0.0515
_backward_Convolution 2268 20984.6953 2.6370 12.5610 9.2525
_mul_scalar 1134 4.0460 0.0020 0.0230 0.0036
_backward_log_softmax 1134 19.2460 0.0090 0.0400 0.0170
mean 3562 68.3340 0.0080 0.0730 0.0192
_backward_mean 1134 12.4210 0.0080 0.1030 0.0110
_backward_FullyConnected 3402 648.0390 0.0280 0.5940 0.1905
WaitForVar 2428 21.6600 0.0060 0.0280 0.0089
broadcast_equal 1294 6.4650 0.0030 0.0110 0.0050
sgd_update 11340 111.9240 0.0020 0.0790 0.0099
_backward_Activation 4536 336.6940 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44432 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3888 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 567 2241.9260 0.3060 73.5850 3.9540
MXNDArrayCreateEx 648 22.1420 0.0310 0.0640 0.0342
MXNDArrayWaitAll 568 12326.0391 5.6130 24.9880 21.7008
MXNDArraySyncCopyToCPU 1216 9603.3818 0.1180 24.1980 7.8975
MXNDArrayGetDType 1216 0.4330 0.0000 0.0010 0.0004
MXNDArrayGetShape 3080 1.4160 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1296 11.3610 0.0070 0.0280 0.0088
MXNDArrayFree 14504 147.9680 0.0000 0.9140 0.0102
MXNet C API Calls 65176 552.5330 487.3580 552.5330 32.5875
MXNet C API Concurrency 130351 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1136 0.5410 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5680 2.9510 0.0000 0.0230 0.0005
MXNDArraySetGradState 5680 2.8840 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1136 0.4820 0.0000 0.0010 0.0004
MXAutogradBackwardEx 568 308.6220 0.5250 0.6810 0.5433
MXNDArrayGetContext 8992 3.8910 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18888 479.7950 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2272 5569.6528 0.7010 3.7500 2.4514
FullyConnected 3888 376.2770 0.0080 0.3430 0.0968
argmax 1296 25.5980 0.0050 0.0480 0.0198
_backward_pick 1136 17.9120 0.0130 0.0260 0.0158
_backward_mul_scalar 1136 4.9120 0.0030 0.0160 0.0043
DeleteVariable 48320 341.6420 0.0010 0.9010 0.0071
Flatten 1296 53.7030 0.0080 0.1120 0.0414
Activation 5184 241.7750 0.0030 1.9440 0.0466
SetValueOp 1136 7.3120 0.0050 0.0180 0.0064
pick 1136 8.9140 0.0060 0.0280 0.0078
Pooling 2592 3640.5139 0.0640 3.1720 1.4045
Convolution 2592 14713.5654 0.2260 13.2050 5.6765
CopyCPU2CPU 1296 7.4200 0.0030 0.0360 0.0057
log_softmax 1136 35.3850 0.0150 0.0550 0.0311
_backward_copy 1136 58.5040 0.0220 0.0760 0.0515
_backward_Convolution 2272 21021.4102 2.6370 12.5610 9.2524
_mul_scalar 1136 4.0540 0.0020 0.0230 0.0036
_backward_log_softmax 1136 19.2780 0.0090 0.0400 0.0170
mean 3568 68.4500 0.0080 0.0730 0.0192
_backward_mean 1136 12.4420 0.0080 0.1030 0.0110
_backward_FullyConnected 3408 649.1120 0.0280 0.5940 0.1905
WaitForVar 2432 21.6950 0.0060 0.0280 0.0089
broadcast_equal 1296 6.4740 0.0030 0.0110 0.0050
sgd_update 11360 112.0740 0.0020 0.0790 0.0099
_backward_Activation 4544 337.3170 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44506 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3894 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 568 2245.8159 0.3060 73.5850 3.9539
MXNDArrayCreateEx 649 22.1750 0.0310 0.0640 0.0342
MXNDArrayWaitAll 569 12347.8779 5.6130 24.9880 21.7010
MXNDArraySyncCopyToCPU 1218 9617.7324 0.1180 24.1980 7.8963
MXNDArrayGetDType 1218 0.4330 0.0000 0.0010 0.0004
MXNDArrayGetShape 3085 1.4190 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1298 11.3780 0.0070 0.0280 0.0088
MXNDArrayFree 14527 148.1910 0.0000 0.9140 0.0102
MXNet C API Calls 65283 552.6400 487.3580 552.6400 32.6410
MXNet C API Concurrency 130565 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1138 0.5420 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5690 2.9560 0.0000 0.0230 0.0005
MXNDArraySetGradState 5690 2.8890 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1138 0.4820 0.0000 0.0010 0.0004
MXAutogradBackwardEx 569 309.1530 0.5250 0.6810 0.5433
MXNDArrayGetContext 9006 3.8990 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18919 480.5640 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2276 5579.3599 0.7010 3.7500 2.4514
FullyConnected 3894 376.8450 0.0080 0.3430 0.0968
argmax 1298 25.6380 0.0050 0.0480 0.0198
_backward_pick 1138 17.9430 0.0130 0.0260 0.0158
_backward_mul_scalar 1138 4.9200 0.0030 0.0160 0.0043
DeleteVariable 48400 342.1580 0.0010 0.9010 0.0071
Flatten 1298 53.7820 0.0080 0.1120 0.0414
Activation 5192 242.1290 0.0030 1.9440 0.0466
SetValueOp 1138 7.3260 0.0050 0.0180 0.0064
pick 1138 8.9290 0.0060 0.0280 0.0078
Pooling 2596 3645.8650 0.0640 3.1720 1.4044
Convolution 2596 14735.7178 0.2260 13.2050 5.6763
CopyCPU2CPU 1298 7.4300 0.0030 0.0360 0.0057
log_softmax 1138 35.4450 0.0150 0.0550 0.0311
_backward_copy 1138 58.6050 0.0220 0.0760 0.0515
_backward_Convolution 2276 21058.4395 2.6370 12.5610 9.2524
_mul_scalar 1138 4.0610 0.0020 0.0230 0.0036
_backward_log_softmax 1138 19.3090 0.0090 0.0400 0.0170
mean 3574 68.5660 0.0080 0.0730 0.0192
_backward_mean 1138 12.4620 0.0080 0.1030 0.0110
_backward_FullyConnected 3414 650.1840 0.0280 0.5940 0.1904
WaitForVar 2436 21.7290 0.0060 0.0280 0.0089
broadcast_equal 1298 6.4830 0.0030 0.0110 0.0050
sgd_update 11380 112.2770 0.0020 0.0790 0.0099
_backward_Activation 4552 337.8780 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44580 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3900 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 569 2249.7261 0.3060 73.5850 3.9538
MXNDArrayCreateEx 650 22.2090 0.0310 0.0640 0.0342
MXNDArrayWaitAll 570 12372.6982 5.6130 24.9880 21.7065
MXNDArraySyncCopyToCPU 1220 9643.8740 0.1180 24.1980 7.9048
MXNDArrayGetDType 1220 0.4340 0.0000 0.0010 0.0004
MXNDArrayGetShape 3090 1.4210 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1300 11.3950 0.0070 0.0280 0.0088
MXNDArrayFree 14550 148.4150 0.0000 0.9140 0.0102
MXNet C API Calls 65390 552.7470 487.3580 552.7470 32.6945
MXNet C API Concurrency 130779 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1140 0.5430 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5700 2.9620 0.0000 0.0230 0.0005
MXNDArraySetGradState 5700 2.8910 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1140 0.4830 0.0000 0.0010 0.0004
MXAutogradBackwardEx 570 309.6860 0.5250 0.6810 0.5433
MXNDArrayGetContext 9020 3.9050 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18950 481.3470 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2280 5590.3501 0.7010 3.7500 2.4519
FullyConnected 3900 377.7280 0.0080 0.3430 0.0969
argmax 1300 25.6910 0.0050 0.0480 0.0198
_backward_pick 1140 17.9840 0.0130 0.0260 0.0158
_backward_mul_scalar 1140 4.9330 0.0030 0.0160 0.0043
DeleteVariable 48480 342.7340 0.0010 0.9010 0.0071
Flatten 1300 53.8760 0.0080 0.1120 0.0414
Activation 5200 242.5540 0.0030 1.9440 0.0466
SetValueOp 1140 7.3440 0.0050 0.0180 0.0064
pick 1140 8.9520 0.0060 0.0280 0.0079
Pooling 2600 3655.4290 0.0640 3.1720 1.4059
Convolution 2600 14774.0752 0.2260 13.2050 5.6823
CopyCPU2CPU 1300 7.4420 0.0030 0.0360 0.0057
log_softmax 1140 35.5230 0.0150 0.0550 0.0312
_backward_copy 1140 58.7250 0.0220 0.0760 0.0515
_backward_Convolution 2280 21101.4160 2.6370 12.5610 9.2550
_mul_scalar 1140 4.0720 0.0020 0.0230 0.0036
_backward_log_softmax 1140 19.3570 0.0090 0.0400 0.0170
mean 3580 68.7200 0.0080 0.0730 0.0192
_backward_mean 1140 12.4930 0.0080 0.1030 0.0110
_backward_FullyConnected 3420 651.8970 0.0280 0.5940 0.1906
WaitForVar 2440 21.7730 0.0060 0.0280 0.0089
broadcast_equal 1300 6.4990 0.0030 0.0110 0.0050
sgd_update 11400 112.5760 0.0020 0.0790 0.0099
_backward_Activation 4560 338.4570 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44654 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3906 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 570 2250.1990 0.3060 73.5850 3.9477
MXNDArrayCreateEx 651 22.2430 0.0310 0.0640 0.0342
MXNDArrayWaitAll 571 12394.3711 5.6130 24.9880 21.7064
MXNDArraySyncCopyToCPU 1222 9658.0674 0.1180 24.1980 7.9035
MXNDArrayGetDType 1222 0.4340 0.0000 0.0010 0.0004
MXNDArrayGetShape 3095 1.4250 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1302 11.4130 0.0070 0.0280 0.0088
MXNDArrayFree 14573 148.6470 0.0000 0.9140 0.0102
MXNet C API Calls 65497 552.8540 487.3580 552.8540 32.7480
MXNet C API Concurrency 130993 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1142 0.5440 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5710 2.9660 0.0000 0.0230 0.0005
MXNDArraySetGradState 5710 2.8970 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1142 0.4850 0.0000 0.0010 0.0004
MXAutogradBackwardEx 571 310.2440 0.5250 0.6810 0.5433
MXNDArrayGetContext 9034 3.9110 0.0000 0.0070 0.0004
MXImperativeInvokeEx 18981 482.1740 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2284 5599.9678 0.7010 3.7500 2.4518
FullyConnected 3906 378.2660 0.0080 0.3430 0.0968
argmax 1302 25.7290 0.0050 0.0480 0.0198
_backward_pick 1142 18.0150 0.0130 0.0260 0.0158
_backward_mul_scalar 1142 4.9420 0.0030 0.0160 0.0043
DeleteVariable 48560 343.2840 0.0010 0.9010 0.0071
Flatten 1302 53.9550 0.0080 0.1120 0.0414
Activation 5208 242.9050 0.0030 1.9440 0.0466
SetValueOp 1142 7.3580 0.0050 0.0180 0.0064
pick 1142 8.9670 0.0060 0.0280 0.0079
Pooling 2604 3660.7749 0.0640 3.1720 1.4058
Convolution 2604 14796.2686 0.2260 13.2050 5.6821
CopyCPU2CPU 1302 7.4530 0.0030 0.0360 0.0057
log_softmax 1142 35.5830 0.0150 0.0550 0.0312
_backward_copy 1142 58.8290 0.0220 0.0760 0.0515
_backward_Convolution 2284 21138.1641 2.6370 12.5610 9.2549
_mul_scalar 1142 4.0800 0.0020 0.0230 0.0036
_backward_log_softmax 1142 19.3890 0.0090 0.0400 0.0170
mean 3586 68.8410 0.0080 0.0730 0.0192
_backward_mean 1142 12.5150 0.0080 0.1030 0.0110
_backward_FullyConnected 3426 653.0050 0.0280 0.5940 0.1906
WaitForVar 2444 21.8060 0.0060 0.0280 0.0089
broadcast_equal 1302 6.5080 0.0030 0.0110 0.0050
sgd_update 11420 112.7760 0.0020 0.0790 0.0099
_backward_Activation 4568 339.0080 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44728 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3912 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 571 2254.0920 0.3060 73.5850 3.9476
MXNDArrayCreateEx 652 22.2760 0.0310 0.0640 0.0342
MXNDArrayWaitAll 572 12416.0254 5.6130 24.9880 21.7063
MXNDArraySyncCopyToCPU 1224 9672.4346 0.1180 24.1980 7.9023
MXNDArrayGetDType 1224 0.4340 0.0000 0.0010 0.0004
MXNDArrayGetShape 3100 1.4270 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1304 11.4310 0.0070 0.0280 0.0088
MXNDArrayFree 14596 148.8760 0.0000 0.9140 0.0102
MXNet C API Calls 65604 552.9610 487.3580 552.9610 32.8015
MXNet C API Concurrency 131207 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1144 0.5440 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5720 2.9710 0.0000 0.0230 0.0005
MXNDArraySetGradState 5720 2.9030 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1144 0.4860 0.0000 0.0010 0.0004
MXAutogradBackwardEx 572 310.8070 0.5250 0.6810 0.5434
MXNDArrayGetContext 9048 3.9190 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19012 482.9490 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2288 5609.5991 0.7010 3.7500 2.4517
FullyConnected 3912 378.8170 0.0080 0.3430 0.0968
argmax 1304 25.7680 0.0050 0.0480 0.0198
_backward_pick 1144 18.0450 0.0130 0.0260 0.0158
_backward_mul_scalar 1144 4.9500 0.0030 0.0160 0.0043
DeleteVariable 48640 343.8120 0.0010 0.9010 0.0071
Flatten 1304 54.0360 0.0080 0.1120 0.0414
Activation 5216 243.2580 0.0030 1.9440 0.0466
SetValueOp 1144 7.3710 0.0050 0.0180 0.0064
pick 1144 8.9830 0.0060 0.0280 0.0079
Pooling 2608 3666.1880 0.0640 3.1720 1.4057
Convolution 2608 14818.3379 0.2260 13.2050 5.6819
CopyCPU2CPU 1304 7.4630 0.0030 0.0360 0.0057
log_softmax 1144 35.6420 0.0150 0.0550 0.0312
_backward_copy 1144 58.9320 0.0220 0.0760 0.0515
_backward_Convolution 2288 21174.9258 2.6370 12.5610 9.2548
_mul_scalar 1144 4.0870 0.0020 0.0230 0.0036
_backward_log_softmax 1144 19.4200 0.0090 0.0400 0.0170
mean 3592 68.9620 0.0080 0.0730 0.0192
_backward_mean 1144 12.5350 0.0080 0.1030 0.0110
_backward_FullyConnected 3432 654.0720 0.0280 0.5940 0.1906
WaitForVar 2448 21.8380 0.0060 0.0280 0.0089
broadcast_equal 1304 6.5170 0.0030 0.0110 0.0050
sgd_update 11440 112.9770 0.0020 0.0790 0.0099
_backward_Activation 4576 339.5990 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44802 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3918 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 572 2258.0090 0.3060 73.5850 3.9476
MXNDArrayCreateEx 653 22.3090 0.0310 0.0640 0.0342
MXNDArrayWaitAll 573 12437.8682 5.6130 24.9880 21.7066
MXNDArraySyncCopyToCPU 1226 9686.8281 0.1180 24.1980 7.9012
MXNDArrayGetDType 1226 0.4350 0.0000 0.0010 0.0004
MXNDArrayGetShape 3105 1.4270 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1306 11.4480 0.0070 0.0280 0.0088
MXNDArrayFree 14619 149.0990 0.0000 0.9140 0.0102
MXNet C API Calls 65711 553.0680 487.3580 553.0680 32.8550
MXNet C API Concurrency 131421 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1146 0.5440 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5730 2.9730 0.0000 0.0230 0.0005
MXNDArraySetGradState 5730 2.9070 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1146 0.4860 0.0000 0.0010 0.0004
MXAutogradBackwardEx 573 311.3370 0.5250 0.6810 0.5433
MXNDArrayGetContext 9062 3.9260 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19043 483.7380 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2292 5619.3408 0.7010 3.7500 2.4517
FullyConnected 3918 379.3500 0.0080 0.3430 0.0968
argmax 1306 25.8060 0.0050 0.0480 0.0198
_backward_pick 1146 18.0760 0.0130 0.0260 0.0158
_backward_mul_scalar 1146 4.9590 0.0030 0.0160 0.0043
DeleteVariable 48720 344.3410 0.0010 0.9010 0.0071
Flatten 1306 54.1140 0.0080 0.1120 0.0414
Activation 5224 243.6080 0.0030 1.9440 0.0466
SetValueOp 1146 7.3840 0.0050 0.0180 0.0064
pick 1146 9.0000 0.0060 0.0280 0.0079
Pooling 2612 3671.5371 0.0640 3.1720 1.4056
Convolution 2612 14840.4961 0.2260 13.2050 5.6817
CopyCPU2CPU 1306 7.4730 0.0030 0.0360 0.0057
log_softmax 1146 35.7010 0.0150 0.0550 0.0312
_backward_copy 1146 59.0350 0.0220 0.0760 0.0515
_backward_Convolution 2292 21211.9746 2.6370 12.5610 9.2548
_mul_scalar 1146 4.0950 0.0020 0.0230 0.0036
_backward_log_softmax 1146 19.4520 0.0090 0.0400 0.0170
mean 3598 69.0770 0.0080 0.0730 0.0192
_backward_mean 1146 12.5570 0.0080 0.1030 0.0110
_backward_FullyConnected 3438 655.1390 0.0280 0.5940 0.1906
WaitForVar 2452 21.8740 0.0060 0.0280 0.0089
broadcast_equal 1306 6.5270 0.0030 0.0110 0.0050
sgd_update 11460 113.1250 0.0020 0.0790 0.0099
_backward_Activation 4584 340.2150 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44876 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3924 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 573 2261.9050 0.3060 73.5850 3.9475
MXNDArrayCreateEx 654 22.3420 0.0310 0.0640 0.0342
MXNDArrayWaitAll 574 12459.7168 5.6130 24.9880 21.7068
MXNDArraySyncCopyToCPU 1228 9701.2080 0.1180 24.1980 7.9000
MXNDArrayGetDType 1228 0.4350 0.0000 0.0010 0.0004
MXNDArrayGetShape 3110 1.4290 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1308 11.4660 0.0070 0.0280 0.0088
MXNDArrayFree 14642 149.3270 0.0000 0.9140 0.0102
MXNet C API Calls 65818 553.1750 487.3580 553.1750 32.9085
MXNet C API Concurrency 131635 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1148 0.5450 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5740 2.9790 0.0000 0.0230 0.0005
MXNDArraySetGradState 5740 2.9100 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1148 0.4860 0.0000 0.0010 0.0004
MXAutogradBackwardEx 574 311.8680 0.5250 0.6810 0.5433
MXNDArrayGetContext 9076 3.9300 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19074 484.5150 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2296 5628.9922 0.7010 3.7500 2.4517
FullyConnected 3924 379.8680 0.0080 0.3430 0.0968
argmax 1308 25.8430 0.0050 0.0480 0.0198
_backward_pick 1148 18.1080 0.0130 0.0260 0.0158
_backward_mul_scalar 1148 4.9670 0.0030 0.0160 0.0043
DeleteVariable 48800 344.8710 0.0010 0.9010 0.0071
Flatten 1308 54.1910 0.0080 0.1120 0.0414
Activation 5232 244.0010 0.0030 1.9440 0.0466
SetValueOp 1148 7.3970 0.0050 0.0180 0.0064
pick 1148 9.0150 0.0060 0.0280 0.0079
Pooling 2616 3676.8840 0.0640 3.1720 1.4055
Convolution 2616 14862.7305 0.2260 13.2050 5.6815
CopyCPU2CPU 1308 7.4830 0.0030 0.0360 0.0057
log_softmax 1148 35.7600 0.0150 0.0550 0.0311
_backward_copy 1148 59.1400 0.0220 0.0760 0.0515
_backward_Convolution 2296 21249.0859 2.6370 12.5610 9.2548
_mul_scalar 1148 4.1020 0.0020 0.0230 0.0036
_backward_log_softmax 1148 19.4840 0.0090 0.0400 0.0170
mean 3604 69.1920 0.0080 0.0730 0.0192
_backward_mean 1148 12.5780 0.0080 0.1030 0.0110
_backward_FullyConnected 3444 656.2030 0.0280 0.5940 0.1905
WaitForVar 2456 21.9090 0.0060 0.0280 0.0089
broadcast_equal 1308 6.5370 0.0030 0.0110 0.0050
sgd_update 11480 113.3280 0.0020 0.0790 0.0099
_backward_Activation 4592 340.7610 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 44950 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3930 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 574 2265.8069 0.3060 73.5850 3.9474
MXNDArrayCreateEx 655 22.3760 0.0310 0.0640 0.0342
MXNDArrayWaitAll 575 12481.3604 5.6130 24.9880 21.7067
MXNDArraySyncCopyToCPU 1230 9715.5840 0.1180 24.1980 7.8988
MXNDArrayGetDType 1230 0.4370 0.0000 0.0010 0.0004
MXNDArrayGetShape 3115 1.4330 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1310 11.4830 0.0070 0.0280 0.0088
MXNDArrayFree 14665 149.5470 0.0000 0.9140 0.0102
MXNet C API Calls 65925 553.2820 487.3580 553.2820 32.9620
MXNet C API Concurrency 131849 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1150 0.5460 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5750 2.9840 0.0000 0.0230 0.0005
MXNDArraySetGradState 5750 2.9160 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1150 0.4870 0.0000 0.0010 0.0004
MXAutogradBackwardEx 575 312.4020 0.5250 0.6810 0.5433
MXNDArrayGetContext 9090 3.9350 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19105 485.2940 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2300 5638.5962 0.7010 3.7500 2.4516
FullyConnected 3930 380.4100 0.0080 0.3430 0.0968
argmax 1310 25.8800 0.0050 0.0480 0.0198
_backward_pick 1150 18.1390 0.0130 0.0260 0.0158
_backward_mul_scalar 1150 4.9750 0.0030 0.0160 0.0043
DeleteVariable 48880 345.3870 0.0010 0.9010 0.0071
Flatten 1310 54.2700 0.0080 0.1120 0.0414
Activation 5240 244.3520 0.0030 1.9440 0.0466
SetValueOp 1150 7.4090 0.0050 0.0180 0.0064
pick 1150 9.0300 0.0060 0.0280 0.0079
Pooling 2620 3682.2749 0.0640 3.1720 1.4054
Convolution 2620 14884.9424 0.2260 13.2050 5.6813
CopyCPU2CPU 1310 7.4930 0.0030 0.0360 0.0057
log_softmax 1150 35.8250 0.0150 0.0550 0.0312
_backward_copy 1150 59.2430 0.0220 0.0760 0.0515
_backward_Convolution 2300 21285.8027 2.6370 12.5610 9.2547
_mul_scalar 1150 4.1090 0.0020 0.0230 0.0036
_backward_log_softmax 1150 19.5150 0.0090 0.0400 0.0170
mean 3610 69.3030 0.0080 0.0730 0.0192
_backward_mean 1150 12.5990 0.0080 0.1030 0.0110
_backward_FullyConnected 3450 657.2650 0.0280 0.5940 0.1905
WaitForVar 2460 21.9430 0.0060 0.0280 0.0089
broadcast_equal 1310 6.5460 0.0030 0.0110 0.0050
sgd_update 11500 113.4740 0.0020 0.0790 0.0099
_backward_Activation 4600 341.3850 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45024 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3936 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 575 2269.7019 0.3060 73.5850 3.9473
MXNDArrayCreateEx 656 22.4100 0.0310 0.0640 0.0342
MXNDArrayWaitAll 576 12503.0898 5.6130 24.9880 21.7068
MXNDArraySyncCopyToCPU 1232 9729.8506 0.1180 24.1980 7.8976
MXNDArrayGetDType 1232 0.4390 0.0000 0.0010 0.0004
MXNDArrayGetShape 3120 1.4360 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1312 11.4990 0.0070 0.0280 0.0088
MXNDArrayFree 14688 149.7770 0.0000 0.9140 0.0102
MXNet C API Calls 66032 553.3890 487.3580 553.3890 33.0155
MXNet C API Concurrency 132063 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1152 0.5460 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5760 2.9870 0.0000 0.0230 0.0005
MXNDArraySetGradState 5760 2.9210 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1152 0.4880 0.0000 0.0010 0.0004
MXAutogradBackwardEx 576 312.9350 0.5250 0.6810 0.5433
MXNDArrayGetContext 9104 3.9430 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19136 486.0700 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2304 5648.3472 0.7010 3.7500 2.4515
FullyConnected 3936 380.9300 0.0080 0.3430 0.0968
argmax 1312 25.9180 0.0050 0.0480 0.0198
_backward_pick 1152 18.1690 0.0130 0.0260 0.0158
_backward_mul_scalar 1152 4.9840 0.0030 0.0160 0.0043
DeleteVariable 48960 345.9250 0.0010 0.9010 0.0071
Flatten 1312 54.3470 0.0080 0.1120 0.0414
Activation 5248 244.7030 0.0030 1.9440 0.0466
SetValueOp 1152 7.4220 0.0050 0.0180 0.0064
pick 1152 9.0450 0.0060 0.0280 0.0079
Pooling 2624 3687.6179 0.0640 3.1720 1.4053
Convolution 2624 14907.0000 0.2260 13.2050 5.6810
CopyCPU2CPU 1312 7.5030 0.0030 0.0360 0.0057
log_softmax 1152 35.8850 0.0150 0.0550 0.0312
_backward_copy 1152 59.3470 0.0220 0.0760 0.0515
_backward_Convolution 2304 21322.5996 2.6370 12.5610 9.2546
_mul_scalar 1152 4.1160 0.0020 0.0230 0.0036
_backward_log_softmax 1152 19.5470 0.0090 0.0400 0.0170
mean 3616 69.4150 0.0080 0.0730 0.0192
_backward_mean 1152 12.6200 0.0080 0.1030 0.0110
_backward_FullyConnected 3456 658.3280 0.0280 0.5940 0.1905
WaitForVar 2464 21.9800 0.0060 0.0280 0.0089
broadcast_equal 1312 6.5550 0.0030 0.0110 0.0050
sgd_update 11520 113.6720 0.0020 0.0790 0.0099
_backward_Activation 4608 341.9630 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45098 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3942 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 576 2273.6121 0.3060 73.5850 3.9472
MXNDArrayCreateEx 657 22.4430 0.0310 0.0640 0.0342
MXNDArrayWaitAll 577 12525.0332 5.6130 24.9880 21.7072
MXNDArraySyncCopyToCPU 1234 9744.2090 0.1180 24.1980 7.8964
MXNDArrayGetDType 1234 0.4390 0.0000 0.0010 0.0004
MXNDArrayGetShape 3125 1.4370 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1314 11.5180 0.0070 0.0280 0.0088
MXNDArrayFree 14711 150.0040 0.0000 0.9140 0.0102
MXNet C API Calls 66139 553.4960 487.3580 553.4960 33.0690
MXNet C API Concurrency 132277 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1154 0.5470 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5770 2.9900 0.0000 0.0230 0.0005
MXNDArraySetGradState 5770 2.9270 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1154 0.4880 0.0000 0.0010 0.0004
MXAutogradBackwardEx 577 313.4990 0.5250 0.6810 0.5433
MXNDArrayGetContext 9118 3.9500 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19167 486.8510 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2308 5658.1802 0.7010 3.7500 2.4516
FullyConnected 3942 381.4520 0.0080 0.3430 0.0968
argmax 1314 25.9550 0.0050 0.0480 0.0198
_backward_pick 1154 18.2000 0.0130 0.0260 0.0158
_backward_mul_scalar 1154 4.9920 0.0030 0.0160 0.0043
DeleteVariable 49040 346.4540 0.0010 0.9010 0.0071
Flatten 1314 54.4280 0.0080 0.1120 0.0414
Activation 5256 245.0550 0.0030 1.9440 0.0466
SetValueOp 1154 7.4350 0.0050 0.0180 0.0064
pick 1154 9.0600 0.0060 0.0280 0.0079
Pooling 2628 3693.0259 0.0640 3.1720 1.4053
Convolution 2628 14929.1631 0.2260 13.2050 5.6808
CopyCPU2CPU 1314 7.5120 0.0030 0.0360 0.0057
log_softmax 1154 35.9440 0.0150 0.0550 0.0311
_backward_copy 1154 59.4490 0.0220 0.0760 0.0515
_backward_Convolution 2308 21359.6797 2.6370 12.5610 9.2546
_mul_scalar 1154 4.1230 0.0020 0.0230 0.0036
_backward_log_softmax 1154 19.5790 0.0090 0.0400 0.0170
mean 3622 69.5290 0.0080 0.0730 0.0192
_backward_mean 1154 12.6410 0.0080 0.1030 0.0110
_backward_FullyConnected 3462 659.3960 0.0280 0.5940 0.1905
WaitForVar 2468 22.0160 0.0060 0.0280 0.0089
broadcast_equal 1314 6.5650 0.0030 0.0110 0.0050
sgd_update 11540 113.8220 0.0020 0.0790 0.0099
_backward_Activation 4616 342.5890 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45172 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3948 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 577 2277.5120 0.3060 73.5850 3.9472
MXNDArrayCreateEx 658 22.4770 0.0310 0.0640 0.0342
MXNDArrayWaitAll 578 12546.9287 5.6130 24.9880 21.7075
MXNDArraySyncCopyToCPU 1236 9758.5938 0.1180 24.1980 7.8953
MXNDArrayGetDType 1236 0.4400 0.0000 0.0010 0.0004
MXNDArrayGetShape 3130 1.4380 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1316 11.5340 0.0070 0.0280 0.0088
MXNDArrayFree 14734 150.2320 0.0000 0.9140 0.0102
MXNet C API Calls 66246 553.6030 487.3580 553.6030 33.1225
MXNet C API Concurrency 132491 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1156 0.5470 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5780 2.9970 0.0000 0.0230 0.0005
MXNDArraySetGradState 5780 2.9320 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1156 0.4890 0.0000 0.0010 0.0004
MXAutogradBackwardEx 578 314.0280 0.5250 0.6810 0.5433
MXNDArrayGetContext 9132 3.9560 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19198 487.6460 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2312 5667.9282 0.7010 3.7500 2.4515
FullyConnected 3948 381.9710 0.0080 0.3430 0.0968
argmax 1316 25.9920 0.0050 0.0480 0.0198
_backward_pick 1156 18.2310 0.0130 0.0260 0.0158
_backward_mul_scalar 1156 5.0010 0.0030 0.0160 0.0043
DeleteVariable 49120 347.0030 0.0010 0.9010 0.0071
Flatten 1316 54.5060 0.0080 0.1120 0.0414
Activation 5264 245.4080 0.0030 1.9440 0.0466
SetValueOp 1156 7.4470 0.0050 0.0180 0.0064
pick 1156 9.0750 0.0060 0.0280 0.0079
Pooling 2632 3698.4260 0.0640 3.1720 1.4052
Convolution 2632 14951.3496 0.2260 13.2050 5.6806
CopyCPU2CPU 1316 7.5210 0.0030 0.0360 0.0057
log_softmax 1156 36.0040 0.0150 0.0550 0.0311
_backward_copy 1156 59.5510 0.0220 0.0760 0.0515
_backward_Convolution 2312 21396.8223 2.6370 12.5610 9.2547
_mul_scalar 1156 4.1300 0.0020 0.0230 0.0036
_backward_log_softmax 1156 19.6100 0.0090 0.0400 0.0170
mean 3628 69.6400 0.0080 0.0730 0.0192
_backward_mean 1156 12.6620 0.0080 0.1030 0.0110
_backward_FullyConnected 3468 660.4600 0.0280 0.5940 0.1904
WaitForVar 2472 22.0510 0.0060 0.0280 0.0089
broadcast_equal 1316 6.5740 0.0030 0.0110 0.0050
sgd_update 11560 113.9680 0.0020 0.0790 0.0099
_backward_Activation 4624 343.1990 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45246 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3954 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 578 2281.4131 0.3060 73.5850 3.9471
MXNDArrayCreateEx 659 22.5100 0.0310 0.0640 0.0342
MXNDArrayWaitAll 579 12568.6787 5.6130 24.9880 21.7076
MXNDArraySyncCopyToCPU 1238 9772.9688 0.1180 24.1980 7.8942
MXNDArrayGetDType 1238 0.4410 0.0000 0.0010 0.0004
MXNDArrayGetShape 3135 1.4430 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1318 11.5510 0.0070 0.0280 0.0088
MXNDArrayFree 14757 150.4600 0.0000 0.9140 0.0102
MXNet C API Calls 66353 553.7100 487.3580 553.7100 33.1760
MXNet C API Concurrency 132705 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1158 0.5470 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5790 3.0050 0.0000 0.0230 0.0005
MXNDArraySetGradState 5790 2.9340 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1158 0.4890 0.0000 0.0010 0.0004
MXAutogradBackwardEx 579 314.5600 0.5250 0.6810 0.5433
MXNDArrayGetContext 9146 3.9590 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19229 488.4450 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2316 5677.7280 0.7010 3.7500 2.4515
FullyConnected 3954 382.5490 0.0080 0.3430 0.0967
argmax 1318 26.0310 0.0050 0.0480 0.0198
_backward_pick 1158 18.2610 0.0130 0.0260 0.0158
_backward_mul_scalar 1158 5.0100 0.0030 0.0160 0.0043
DeleteVariable 49200 347.5370 0.0010 0.9010 0.0071
Flatten 1318 54.5850 0.0080 0.1120 0.0414
Activation 5272 245.7620 0.0030 1.9440 0.0466
SetValueOp 1158 7.4590 0.0050 0.0180 0.0064
pick 1158 9.0900 0.0060 0.0280 0.0078
Pooling 2636 3703.7571 0.0640 3.1720 1.4051
Convolution 2636 14973.5107 0.2260 13.2050 5.6804
CopyCPU2CPU 1318 7.5300 0.0030 0.0360 0.0057
log_softmax 1158 36.0630 0.0150 0.0550 0.0311
_backward_copy 1158 59.6530 0.0220 0.0760 0.0515
_backward_Convolution 2316 21433.6504 2.6370 12.5610 9.2546
_mul_scalar 1158 4.1370 0.0020 0.0230 0.0036
_backward_log_softmax 1158 19.6420 0.0090 0.0400 0.0170
mean 3634 69.7540 0.0080 0.0730 0.0192
_backward_mean 1158 12.6830 0.0080 0.1030 0.0110
_backward_FullyConnected 3474 661.5230 0.0280 0.5940 0.1904
WaitForVar 2476 22.0860 0.0060 0.0280 0.0089
broadcast_equal 1318 6.5830 0.0030 0.0110 0.0050
sgd_update 11580 114.1650 0.0020 0.0790 0.0099
_backward_Activation 4632 343.7440 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45320 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3960 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 579 2285.3201 0.3060 73.5850 3.9470
MXNDArrayCreateEx 660 22.5440 0.0310 0.0640 0.0342
MXNDArrayWaitAll 580 12590.3623 5.6130 24.9880 21.7075
MXNDArraySyncCopyToCPU 1240 9787.2676 0.1180 24.1980 7.8930
MXNDArrayGetDType 1240 0.4420 0.0000 0.0010 0.0004
MXNDArrayGetShape 3140 1.4470 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1320 11.5680 0.0070 0.0280 0.0088
MXNDArrayFree 14780 150.6820 0.0000 0.9140 0.0102
MXNet C API Calls 66460 553.8170 487.3580 553.8170 33.2295
MXNet C API Concurrency 132919 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1160 0.5480 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5800 3.0090 0.0000 0.0230 0.0005
MXNDArraySetGradState 5800 2.9370 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1160 0.4890 0.0000 0.0010 0.0004
MXAutogradBackwardEx 580 315.0900 0.5250 0.6810 0.5433
MXNDArrayGetContext 9160 3.9670 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19260 489.2440 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2320 5687.4282 0.7010 3.7500 2.4515
FullyConnected 3960 383.0680 0.0080 0.3430 0.0967
argmax 1320 26.0680 0.0050 0.0480 0.0197
_backward_pick 1160 18.2920 0.0130 0.0260 0.0158
_backward_mul_scalar 1160 5.0180 0.0030 0.0160 0.0043
DeleteVariable 49280 348.0690 0.0010 0.9010 0.0071
Flatten 1320 54.6640 0.0080 0.1120 0.0414
Activation 5280 246.1140 0.0030 1.9440 0.0466
SetValueOp 1160 7.4710 0.0050 0.0180 0.0064
pick 1160 9.1050 0.0060 0.0280 0.0078
Pooling 2640 3709.1079 0.0640 3.1720 1.4050
Convolution 2640 14995.5381 0.2260 13.2050 5.6801
CopyCPU2CPU 1320 7.5390 0.0030 0.0360 0.0057
log_softmax 1160 36.1230 0.0150 0.0550 0.0311
_backward_copy 1160 59.7560 0.0220 0.0760 0.0515
_backward_Convolution 2320 21470.4258 2.6370 12.5610 9.2545
_mul_scalar 1160 4.1440 0.0020 0.0230 0.0036
_backward_log_softmax 1160 19.6740 0.0090 0.0400 0.0170
mean 3640 69.8660 0.0080 0.0730 0.0192
_backward_mean 1160 12.7040 0.0080 0.1030 0.0110
_backward_FullyConnected 3480 662.5860 0.0280 0.5940 0.1904
WaitForVar 2480 22.1220 0.0060 0.0280 0.0089
broadcast_equal 1320 6.5920 0.0030 0.0110 0.0050
sgd_update 11600 114.3640 0.0020 0.0790 0.0099
_backward_Activation 4640 344.2940 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45394 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3966 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 580 2289.2151 0.3060 73.5850 3.9469
MXNDArrayCreateEx 661 22.5770 0.0310 0.0640 0.0342
MXNDArrayWaitAll 581 12612.2148 5.6130 24.9880 21.7078
MXNDArraySyncCopyToCPU 1242 9801.7363 0.1180 24.1980 7.8919
MXNDArrayGetDType 1242 0.4440 0.0000 0.0010 0.0004
MXNDArrayGetShape 3145 1.4490 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1322 11.5860 0.0070 0.0280 0.0088
MXNDArrayFree 14803 150.9040 0.0000 0.9140 0.0102
MXNet C API Calls 66567 553.9240 487.3580 553.9240 33.2830
MXNet C API Concurrency 133133 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1162 0.5480 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5810 3.0140 0.0000 0.0230 0.0005
MXNDArraySetGradState 5810 2.9440 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1162 0.4890 0.0000 0.0010 0.0004
MXAutogradBackwardEx 581 315.6270 0.5250 0.6810 0.5432
MXNDArrayGetContext 9174 3.9720 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19291 490.0130 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2324 5697.3662 0.7010 3.7500 2.4515
FullyConnected 3966 383.5890 0.0080 0.3430 0.0967
argmax 1322 26.1060 0.0050 0.0480 0.0197
_backward_pick 1162 18.3240 0.0130 0.0260 0.0158
_backward_mul_scalar 1162 5.0260 0.0030 0.0160 0.0043
DeleteVariable 49360 348.5900 0.0010 0.9010 0.0071
Flatten 1322 54.7430 0.0080 0.1120 0.0414
Activation 5288 246.4640 0.0030 1.9440 0.0466
SetValueOp 1162 7.4830 0.0050 0.0180 0.0064
pick 1162 9.1200 0.0060 0.0280 0.0078
Pooling 2644 3714.4580 0.0640 3.1720 1.4049
Convolution 2644 15017.6768 0.2260 13.2050 5.6799
CopyCPU2CPU 1322 7.5490 0.0030 0.0360 0.0057
log_softmax 1162 36.1820 0.0150 0.0550 0.0311
_backward_copy 1162 59.8580 0.0220 0.0760 0.0515
_backward_Convolution 2324 21507.4844 2.6370 12.5610 9.2545
_mul_scalar 1162 4.1510 0.0020 0.0230 0.0036
_backward_log_softmax 1162 19.7050 0.0090 0.0400 0.0170
mean 3646 69.9860 0.0080 0.0730 0.0192
_backward_mean 1162 12.7250 0.0080 0.1030 0.0110
_backward_FullyConnected 3486 663.6550 0.0280 0.5940 0.1904
WaitForVar 2484 22.1570 0.0060 0.0280 0.0089
broadcast_equal 1322 6.6020 0.0030 0.0110 0.0050
sgd_update 11620 114.5650 0.0020 0.0790 0.0099
_backward_Activation 4648 344.8540 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45468 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3972 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 581 2293.1040 0.3060 73.5850 3.9468
MXNDArrayCreateEx 662 22.6110 0.0310 0.0640 0.0342
MXNDArrayWaitAll 582 12634.1250 5.6130 24.9880 21.7081
MXNDArraySyncCopyToCPU 1244 9816.1172 0.1180 24.1980 7.8908
MXNDArrayGetDType 1244 0.4440 0.0000 0.0010 0.0004
MXNDArrayGetShape 3150 1.4510 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1324 11.6030 0.0070 0.0280 0.0088
MXNDArrayFree 14826 151.1240 0.0000 0.9140 0.0102
MXNet C API Calls 66674 554.0310 487.3580 554.0310 33.3365
MXNet C API Concurrency 133347 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1164 0.5490 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5820 3.0200 0.0000 0.0230 0.0005
MXNDArraySetGradState 5820 2.9460 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1164 0.4900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 582 316.1860 0.5250 0.6810 0.5433
MXNDArrayGetContext 9188 3.9810 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19322 490.7840 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2328 5707.1279 0.7010 3.7500 2.4515
FullyConnected 3972 384.1380 0.0080 0.3430 0.0967
argmax 1324 26.1440 0.0050 0.0480 0.0197
_backward_pick 1164 18.3550 0.0130 0.0260 0.0158
_backward_mul_scalar 1164 5.0340 0.0030 0.0160 0.0043
DeleteVariable 49440 349.1200 0.0010 0.9010 0.0071
Flatten 1324 54.8240 0.0080 0.1120 0.0414
Activation 5296 246.8120 0.0030 1.9440 0.0466
SetValueOp 1164 7.4940 0.0050 0.0180 0.0064
pick 1164 9.1350 0.0060 0.0280 0.0078
Pooling 2648 3719.8459 0.0640 3.1720 1.4048
Convolution 2648 15039.9062 0.2260 13.2050 5.6797
CopyCPU2CPU 1324 7.5580 0.0030 0.0360 0.0057
log_softmax 1164 36.2420 0.0150 0.0550 0.0311
_backward_copy 1164 59.9610 0.0220 0.0760 0.0515
_backward_Convolution 2328 21544.6133 2.6370 12.5610 9.2546
_mul_scalar 1164 4.1590 0.0020 0.0230 0.0036
_backward_log_softmax 1164 19.7360 0.0090 0.0400 0.0170
mean 3652 70.1020 0.0080 0.0730 0.0192
_backward_mean 1164 12.7460 0.0080 0.1030 0.0110
_backward_FullyConnected 3492 664.7270 0.0280 0.5940 0.1904
WaitForVar 2488 22.1920 0.0060 0.0280 0.0089
broadcast_equal 1324 6.6100 0.0030 0.0110 0.0050
sgd_update 11640 114.7630 0.0020 0.0790 0.0099
_backward_Activation 4656 345.3810 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45542 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3978 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 582 2297.0061 0.3060 73.5850 3.9467
MXNDArrayCreateEx 663 22.6440 0.0310 0.0640 0.0342
MXNDArrayWaitAll 583 12655.7998 5.6130 24.9880 21.7081
MXNDArraySyncCopyToCPU 1246 9830.5020 0.1180 24.1980 7.8896
MXNDArrayGetDType 1246 0.4450 0.0000 0.0010 0.0004
MXNDArrayGetShape 3155 1.4540 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1326 11.6200 0.0070 0.0280 0.0088
MXNDArrayFree 14849 151.3510 0.0000 0.9140 0.0102
MXNet C API Calls 66781 554.1380 487.3580 554.1380 33.3900
MXNet C API Concurrency 133561 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1166 0.5500 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5830 3.0220 0.0000 0.0230 0.0005
MXNDArraySetGradState 5830 2.9520 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1166 0.4900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 583 316.7190 0.5250 0.6810 0.5433
MXNDArrayGetContext 9202 3.9860 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19353 491.5750 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2332 5716.8359 0.7010 3.7500 2.4515
FullyConnected 3978 384.6680 0.0080 0.3430 0.0967
argmax 1326 26.1810 0.0050 0.0480 0.0197
_backward_pick 1166 18.3860 0.0130 0.0260 0.0158
_backward_mul_scalar 1166 5.0430 0.0030 0.0160 0.0043
DeleteVariable 49520 349.6490 0.0010 0.9010 0.0071
Flatten 1326 54.9010 0.0080 0.1120 0.0414
Activation 5304 247.1630 0.0030 1.9440 0.0466
SetValueOp 1166 7.5080 0.0050 0.0180 0.0064
pick 1166 9.1500 0.0060 0.0280 0.0078
Pooling 2652 3725.2029 0.0640 3.1720 1.4047
Convolution 2652 15062.0527 0.2260 13.2050 5.6795
CopyCPU2CPU 1326 7.5670 0.0030 0.0360 0.0057
log_softmax 1166 36.3010 0.0150 0.0550 0.0311
_backward_copy 1166 60.0640 0.0220 0.0760 0.0515
_backward_Convolution 2332 21581.3730 2.6370 12.5610 9.2544
_mul_scalar 1166 4.1660 0.0020 0.0230 0.0036
_backward_log_softmax 1166 19.7690 0.0090 0.0400 0.0170
mean 3658 70.2120 0.0080 0.0730 0.0192
_backward_mean 1166 12.7670 0.0080 0.1030 0.0109
_backward_FullyConnected 3498 665.7870 0.0280 0.5940 0.1903
WaitForVar 2492 22.2270 0.0060 0.0280 0.0089
broadcast_equal 1326 6.6190 0.0030 0.0110 0.0050
sgd_update 11660 114.9610 0.0020 0.0790 0.0099
_backward_Activation 4664 345.9460 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45616 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3984 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 583 2300.9150 0.3060 73.5850 3.9467
MXNDArrayCreateEx 664 22.6790 0.0310 0.0640 0.0342
MXNDArrayWaitAll 584 12677.4463 5.6130 24.9880 21.7080
MXNDArraySyncCopyToCPU 1248 9844.6650 0.1180 24.1980 7.8884
MXNDArrayGetDType 1248 0.4450 0.0000 0.0010 0.0004
MXNDArrayGetShape 3160 1.4550 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1328 11.6370 0.0070 0.0280 0.0088
MXNDArrayFree 14872 151.5840 0.0000 0.9140 0.0102
MXNet C API Calls 66888 554.2450 487.3580 554.2450 33.4435
MXNet C API Concurrency 133775 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1168 0.5520 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5840 3.0260 0.0000 0.0230 0.0005
MXNDArraySetGradState 5840 2.9540 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1168 0.4900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 584 317.2510 0.5250 0.6810 0.5432
MXNDArrayGetContext 9216 3.9920 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19384 492.3640 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2336 5726.4009 0.7010 3.7500 2.4514
FullyConnected 3984 385.2210 0.0080 0.3430 0.0967
argmax 1328 26.2190 0.0050 0.0480 0.0197
_backward_pick 1168 18.4160 0.0130 0.0260 0.0158
_backward_mul_scalar 1168 5.0520 0.0030 0.0160 0.0043
DeleteVariable 49600 350.1730 0.0010 0.9010 0.0071
Flatten 1328 54.9790 0.0080 0.1120 0.0414
Activation 5312 247.5150 0.0030 1.9440 0.0466
SetValueOp 1168 7.5210 0.0050 0.0180 0.0064
pick 1168 9.1640 0.0060 0.0280 0.0078
Pooling 2656 3730.5471 0.0640 3.1720 1.4046
Convolution 2656 15084.1299 0.2260 13.2050 5.6793
CopyCPU2CPU 1328 7.5770 0.0030 0.0360 0.0057
log_softmax 1168 36.3610 0.0150 0.0550 0.0311
_backward_copy 1168 60.1670 0.0220 0.0760 0.0515
_backward_Convolution 2336 21618.1582 2.6370 12.5610 9.2543
_mul_scalar 1168 4.1730 0.0020 0.0230 0.0036
_backward_log_softmax 1168 19.8000 0.0090 0.0400 0.0170
mean 3664 70.3240 0.0080 0.0730 0.0192
_backward_mean 1168 12.7880 0.0080 0.1030 0.0109
_backward_FullyConnected 3504 666.8490 0.0280 0.5940 0.1903
WaitForVar 2496 22.2610 0.0060 0.0280 0.0089
broadcast_equal 1328 6.6280 0.0030 0.0110 0.0050
sgd_update 11680 115.1580 0.0020 0.0790 0.0099
_backward_Activation 4672 346.4920 0.0070 0.3900 0.0742
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45690 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3990 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 584 2304.8130 0.3060 73.5850 3.9466
MXNDArrayCreateEx 665 22.7120 0.0310 0.0640 0.0342
MXNDArrayWaitAll 585 12699.2842 5.6130 24.9880 21.7082
MXNDArraySyncCopyToCPU 1250 9859.0186 0.1180 24.1980 7.8872
MXNDArrayGetDType 1250 0.4460 0.0000 0.0010 0.0004
MXNDArrayGetShape 3165 1.4560 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1330 11.6540 0.0070 0.0280 0.0088
MXNDArrayFree 14895 151.8170 0.0000 0.9140 0.0102
MXNet C API Calls 66995 554.3520 487.3580 554.3520 33.4970
MXNet C API Concurrency 133989 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1170 0.5530 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5850 3.0310 0.0000 0.0230 0.0005
MXNDArraySetGradState 5850 2.9570 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1170 0.4900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 585 317.7820 0.5250 0.6810 0.5432
MXNDArrayGetContext 9230 4.0000 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19415 493.1400 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2340 5736.1318 0.7010 3.7500 2.4513
FullyConnected 3990 385.7670 0.0080 0.3430 0.0967
argmax 1330 26.2560 0.0050 0.0480 0.0197
_backward_pick 1170 18.4470 0.0130 0.0260 0.0158
_backward_mul_scalar 1170 5.0610 0.0030 0.0160 0.0043
DeleteVariable 49680 350.7180 0.0010 0.9010 0.0071
Flatten 1330 55.0560 0.0080 0.1120 0.0414
Activation 5320 247.8670 0.0030 1.9440 0.0466
SetValueOp 1170 7.5320 0.0050 0.0180 0.0064
pick 1170 9.1790 0.0060 0.0280 0.0078
Pooling 2660 3735.9089 0.0640 3.1720 1.4045
Convolution 2660 15106.2734 0.2260 13.2050 5.6790
CopyCPU2CPU 1330 7.5860 0.0030 0.0360 0.0057
log_softmax 1170 36.4200 0.0150 0.0550 0.0311
_backward_copy 1170 60.2710 0.0220 0.0760 0.0515
_backward_Convolution 2340 21655.2012 2.6370 12.5610 9.2544
_mul_scalar 1170 4.1800 0.0020 0.0230 0.0036
_backward_log_softmax 1170 19.8310 0.0090 0.0400 0.0169
mean 3670 70.4380 0.0080 0.0730 0.0192
_backward_mean 1170 12.8100 0.0080 0.1030 0.0109
_backward_FullyConnected 3510 667.9150 0.0280 0.5940 0.1903
WaitForVar 2500 22.2940 0.0060 0.0280 0.0089
broadcast_equal 1330 6.6370 0.0030 0.0110 0.0050
sgd_update 11700 115.3600 0.0020 0.0790 0.0099
_backward_Activation 4680 347.0210 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45764 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 3996 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 585 2308.7161 0.3060 73.5850 3.9465
MXNDArrayCreateEx 666 22.7450 0.0310 0.0640 0.0342
MXNDArrayWaitAll 586 12721.1836 5.6130 24.9880 21.7085
MXNDArraySyncCopyToCPU 1252 9873.4043 0.1180 24.1980 7.8861
MXNDArrayGetDType 1252 0.4460 0.0000 0.0010 0.0004
MXNDArrayGetShape 3170 1.4580 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1332 11.6710 0.0070 0.0280 0.0088
MXNDArrayFree 14918 152.0460 0.0000 0.9140 0.0102
MXNet C API Calls 67102 554.4590 487.3580 554.4590 33.5505
MXNet C API Concurrency 134203 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1172 0.5530 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5860 3.0390 0.0000 0.0230 0.0005
MXNDArraySetGradState 5860 2.9600 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1172 0.4900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 586 318.3110 0.5250 0.6810 0.5432
MXNDArrayGetContext 9244 4.0060 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19446 493.9060 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2344 5745.8481 0.7010 3.7500 2.4513
FullyConnected 3996 386.2860 0.0080 0.3430 0.0967
argmax 1332 26.3330 0.0050 0.0480 0.0198
_backward_pick 1172 18.4780 0.0130 0.0260 0.0158
_backward_mul_scalar 1172 5.0700 0.0030 0.0160 0.0043
DeleteVariable 49760 351.2350 0.0010 0.9010 0.0071
Flatten 1332 55.1330 0.0080 0.1120 0.0414
Activation 5328 248.2190 0.0030 1.9440 0.0466
SetValueOp 1172 7.5440 0.0050 0.0180 0.0064
pick 1172 9.1950 0.0060 0.0280 0.0078
Pooling 2664 3741.2549 0.0640 3.1720 1.4044
Convolution 2664 15128.4941 0.2260 13.2050 5.6789
CopyCPU2CPU 1332 7.5960 0.0030 0.0360 0.0057
log_softmax 1172 36.4790 0.0150 0.0550 0.0311
_backward_copy 1172 60.3720 0.0220 0.0760 0.0515
_backward_Convolution 2344 21692.3340 2.6370 12.5610 9.2544
_mul_scalar 1172 4.1870 0.0020 0.0230 0.0036
_backward_log_softmax 1172 19.8630 0.0090 0.0400 0.0169
mean 3676 70.5490 0.0080 0.0730 0.0192
_backward_mean 1172 12.8310 0.0080 0.1030 0.0109
_backward_FullyConnected 3516 668.9790 0.0280 0.5940 0.1903
WaitForVar 2504 22.3280 0.0060 0.0280 0.0089
broadcast_equal 1332 6.6470 0.0030 0.0110 0.0050
sgd_update 11720 115.5600 0.0020 0.0790 0.0099
_backward_Activation 4688 347.5710 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45838 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4002 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 586 2312.6311 0.3060 73.5850 3.9465
MXNDArrayCreateEx 667 22.7770 0.0310 0.0640 0.0341
MXNDArrayWaitAll 587 12742.8779 5.6130 24.9880 21.7085
MXNDArraySyncCopyToCPU 1254 9887.7959 0.1180 24.1980 7.8850
MXNDArrayGetDType 1254 0.4470 0.0000 0.0010 0.0004
MXNDArrayGetShape 3175 1.4580 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1334 11.6880 0.0070 0.0280 0.0088
MXNDArrayFree 14941 152.2770 0.0000 0.9140 0.0102
MXNet C API Calls 67209 554.5660 487.3580 554.5660 33.6040
MXNet C API Concurrency 134417 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1174 0.5540 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5870 3.0430 0.0000 0.0230 0.0005
MXNDArraySetGradState 5870 2.9660 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1174 0.4900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 587 318.8630 0.5250 0.6810 0.5432
MXNDArrayGetContext 9258 4.0110 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19477 494.6750 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2348 5755.5659 0.7010 3.7500 2.4513
FullyConnected 4002 386.8350 0.0080 0.3430 0.0967
argmax 1334 26.3700 0.0050 0.0480 0.0198
_backward_pick 1174 18.5090 0.0130 0.0260 0.0158
_backward_mul_scalar 1174 5.0780 0.0030 0.0160 0.0043
DeleteVariable 49840 351.7910 0.0010 0.9010 0.0071
Flatten 1334 55.2120 0.0080 0.1120 0.0414
Activation 5336 248.5710 0.0030 1.9440 0.0466
SetValueOp 1174 7.5570 0.0050 0.0180 0.0064
pick 1174 9.2100 0.0060 0.0280 0.0078
Pooling 2668 3746.6399 0.0640 3.1720 1.4043
Convolution 2668 15150.6523 0.2260 13.2050 5.6787
CopyCPU2CPU 1334 7.6050 0.0030 0.0360 0.0057
log_softmax 1174 36.5380 0.0150 0.0550 0.0311
_backward_copy 1174 60.4760 0.0220 0.0760 0.0515
_backward_Convolution 2348 21729.0586 2.6370 12.5610 9.2543
_mul_scalar 1174 4.1940 0.0020 0.0230 0.0036
_backward_log_softmax 1174 19.8950 0.0090 0.0400 0.0169
mean 3682 70.6600 0.0080 0.0730 0.0192
_backward_mean 1174 12.8530 0.0080 0.1030 0.0109
_backward_FullyConnected 3522 670.0450 0.0280 0.5940 0.1902
WaitForVar 2508 22.3630 0.0060 0.0280 0.0089
broadcast_equal 1334 6.6560 0.0030 0.0110 0.0050
sgd_update 11740 115.7580 0.0020 0.0790 0.0099
_backward_Activation 4696 348.1420 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45912 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4008 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 587 2316.5339 0.3060 73.5850 3.9464
MXNDArrayCreateEx 668 22.8100 0.0310 0.0640 0.0341
MXNDArrayWaitAll 588 12764.5645 5.6130 24.9880 21.7084
MXNDArraySyncCopyToCPU 1256 9902.1064 0.1180 24.1980 7.8838
MXNDArrayGetDType 1256 0.4490 0.0000 0.0010 0.0004
MXNDArrayGetShape 3180 1.4610 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1336 11.7040 0.0070 0.0280 0.0088
MXNDArrayFree 14964 152.5010 0.0000 0.9140 0.0102
MXNet C API Calls 67316 554.6730 487.3580 554.6730 33.6575
MXNet C API Concurrency 134631 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1176 0.5540 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5880 3.0470 0.0000 0.0230 0.0005
MXNDArraySetGradState 5880 2.9720 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1176 0.4900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 588 319.4010 0.5250 0.6810 0.5432
MXNDArrayGetContext 9272 4.0180 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19508 495.4440 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2352 5765.2041 0.7010 3.7500 2.4512
FullyConnected 4008 387.3740 0.0080 0.3430 0.0967
argmax 1336 26.4070 0.0050 0.0480 0.0198
_backward_pick 1176 18.5380 0.0130 0.0260 0.0158
_backward_mul_scalar 1176 5.0860 0.0030 0.0160 0.0043
DeleteVariable 49920 352.3140 0.0010 0.9010 0.0071
Flatten 1336 55.2890 0.0080 0.1120 0.0414
Activation 5344 248.9200 0.0030 1.9440 0.0466
SetValueOp 1176 7.5680 0.0050 0.0180 0.0064
pick 1176 9.2250 0.0060 0.0280 0.0078
Pooling 2672 3752.0039 0.0640 3.1720 1.4042
Convolution 2672 15172.6924 0.2260 13.2050 5.6784
CopyCPU2CPU 1336 7.6140 0.0030 0.0360 0.0057
log_softmax 1176 36.5970 0.0150 0.0550 0.0311
_backward_copy 1176 60.5790 0.0220 0.0760 0.0515
_backward_Convolution 2352 21765.8828 2.6370 12.5610 9.2542
_mul_scalar 1176 4.2010 0.0020 0.0230 0.0036
_backward_log_softmax 1176 19.9270 0.0090 0.0400 0.0169
mean 3688 70.7710 0.0080 0.0730 0.0192
_backward_mean 1176 12.8740 0.0080 0.1030 0.0109
_backward_FullyConnected 3528 671.1140 0.0280 0.5940 0.1902
WaitForVar 2512 22.3980 0.0060 0.0280 0.0089
broadcast_equal 1336 6.6650 0.0030 0.0110 0.0050
sgd_update 11760 115.9110 0.0020 0.0790 0.0099
_backward_Activation 4704 348.7540 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 45986 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4014 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 588 2320.4460 0.3060 73.5850 3.9463
MXNDArrayCreateEx 669 22.8430 0.0310 0.0640 0.0341
MXNDArrayWaitAll 589 12786.3760 5.6130 24.9880 21.7086
MXNDArraySyncCopyToCPU 1258 9916.4277 0.1180 24.1980 7.8827
MXNDArrayGetDType 1258 0.4500 0.0000 0.0010 0.0004
MXNDArrayGetShape 3185 1.4640 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1338 11.7210 0.0070 0.0280 0.0088
MXNDArrayFree 14987 152.7290 0.0000 0.9140 0.0102
MXNet C API Calls 67423 554.7800 487.3580 554.7800 33.7110
MXNet C API Concurrency 134845 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1178 0.5550 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5890 3.0520 0.0000 0.0230 0.0005
MXNDArraySetGradState 5890 2.9780 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1178 0.4900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 589 319.9340 0.5250 0.6810 0.5432
MXNDArrayGetContext 9286 4.0250 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19539 496.2190 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2356 5774.8071 0.7010 3.7500 2.4511
FullyConnected 4014 387.8910 0.0080 0.3430 0.0966
argmax 1338 26.4450 0.0050 0.0480 0.0198
_backward_pick 1178 18.5690 0.0130 0.0260 0.0158
_backward_mul_scalar 1178 5.0940 0.0030 0.0160 0.0043
DeleteVariable 50000 352.8470 0.0010 0.9010 0.0071
Flatten 1338 55.3670 0.0080 0.1120 0.0414
Activation 5352 249.2710 0.0030 1.9440 0.0466
SetValueOp 1178 7.5810 0.0050 0.0180 0.0064
pick 1178 9.2390 0.0060 0.0280 0.0078
Pooling 2676 3757.3469 0.0640 3.1720 1.4041
Convolution 2676 15194.8818 0.2260 13.2050 5.6782
CopyCPU2CPU 1338 7.6230 0.0030 0.0360 0.0057
log_softmax 1178 36.6550 0.0150 0.0550 0.0311
_backward_copy 1178 60.6800 0.0220 0.0760 0.0515
_backward_Convolution 2356 21802.9121 2.6370 12.5610 9.2542
_mul_scalar 1178 4.2080 0.0020 0.0230 0.0036
_backward_log_softmax 1178 19.9590 0.0090 0.0400 0.0169
mean 3694 70.8880 0.0080 0.0730 0.0192
_backward_mean 1178 12.8950 0.0080 0.1030 0.0109
_backward_FullyConnected 3534 672.1800 0.0280 0.5940 0.1902
WaitForVar 2516 22.4310 0.0060 0.0280 0.0089
broadcast_equal 1338 6.6740 0.0030 0.0110 0.0050
sgd_update 11780 116.1160 0.0020 0.0790 0.0099
_backward_Activation 4712 349.2890 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46060 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4020 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 589 2324.3530 0.3060 73.5850 3.9463
MXNDArrayCreateEx 670 22.8760 0.0310 0.0640 0.0341
MXNDArrayWaitAll 590 12808.2422 5.6130 24.9880 21.7089
MXNDArraySyncCopyToCPU 1260 9930.8174 0.1180 24.1980 7.8816
MXNDArrayGetDType 1260 0.4520 0.0000 0.0010 0.0004
MXNDArrayGetShape 3190 1.4660 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1340 11.7390 0.0070 0.0280 0.0088
MXNDArrayFree 15010 152.9500 0.0000 0.9140 0.0102
MXNet C API Calls 67530 554.8870 487.3580 554.8870 33.7645
MXNet C API Concurrency 135059 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1180 0.5560 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5900 3.0560 0.0000 0.0230 0.0005
MXNDArraySetGradState 5900 2.9860 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1180 0.4900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 590 320.4660 0.5250 0.6810 0.5432
MXNDArrayGetContext 9300 4.0320 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19570 497.0050 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2360 5784.5591 0.7010 3.7500 2.4511
FullyConnected 4020 388.4370 0.0080 0.3430 0.0966
argmax 1340 26.4830 0.0050 0.0480 0.0198
_backward_pick 1180 18.5990 0.0130 0.0260 0.0158
_backward_mul_scalar 1180 5.1040 0.0030 0.0160 0.0043
DeleteVariable 50080 353.3580 0.0010 0.9010 0.0071
Flatten 1340 55.4450 0.0080 0.1120 0.0414
Activation 5360 249.6550 0.0030 1.9440 0.0466
SetValueOp 1180 7.5930 0.0050 0.0180 0.0064
pick 1180 9.2550 0.0060 0.0280 0.0078
Pooling 2680 3762.7620 0.0640 3.1720 1.4040
Convolution 2680 15217.0576 0.2260 13.2050 5.6780
CopyCPU2CPU 1340 7.6320 0.0030 0.0360 0.0057
log_softmax 1180 36.7140 0.0150 0.0550 0.0311
_backward_copy 1180 60.7840 0.0220 0.0760 0.0515
_backward_Convolution 2360 21839.9902 2.6370 12.5610 9.2542
_mul_scalar 1180 4.2160 0.0020 0.0230 0.0036
_backward_log_softmax 1180 19.9910 0.0090 0.0400 0.0169
mean 3700 70.9990 0.0080 0.0730 0.0192
_backward_mean 1180 12.9160 0.0080 0.1030 0.0109
_backward_FullyConnected 3540 673.2450 0.0280 0.5940 0.1902
WaitForVar 2520 22.4650 0.0060 0.0280 0.0089
broadcast_equal 1340 6.6830 0.0030 0.0110 0.0050
sgd_update 11800 116.3150 0.0020 0.0790 0.0099
_backward_Activation 4720 349.8360 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46134 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4026 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 590 2328.2671 0.3060 73.5850 3.9462
MXNDArrayCreateEx 671 22.9080 0.0310 0.0640 0.0341
MXNDArrayWaitAll 591 12829.9346 5.6130 24.9880 21.7089
MXNDArraySyncCopyToCPU 1262 9945.1699 0.1180 24.1980 7.8805
MXNDArrayGetDType 1262 0.4540 0.0000 0.0010 0.0004
MXNDArrayGetShape 3195 1.4690 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1342 11.7560 0.0070 0.0280 0.0088
MXNDArrayFree 15033 153.1740 0.0000 0.9140 0.0102
MXNet C API Calls 67637 554.9940 487.3580 554.9940 33.8180
MXNet C API Concurrency 135273 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1182 0.5570 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5910 3.0630 0.0000 0.0230 0.0005
MXNDArraySetGradState 5910 2.9900 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1182 0.4910 0.0000 0.0010 0.0004
MXAutogradBackwardEx 591 321.0190 0.5250 0.6810 0.5432
MXNDArrayGetContext 9314 4.0360 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19601 497.8020 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2364 5794.2769 0.7010 3.7500 2.4510
FullyConnected 4026 389.0250 0.0080 0.3430 0.0966
argmax 1342 26.5230 0.0050 0.0480 0.0198
_backward_pick 1182 18.6330 0.0130 0.0260 0.0158
_backward_mul_scalar 1182 5.1130 0.0030 0.0160 0.0043
DeleteVariable 50160 353.8910 0.0010 0.9010 0.0071
Flatten 1342 55.5220 0.0080 0.1120 0.0414
Activation 5368 250.0510 0.0030 1.9440 0.0466
SetValueOp 1182 7.6040 0.0050 0.0180 0.0064
pick 1182 9.2720 0.0060 0.0280 0.0078
Pooling 2684 3768.0891 0.0640 3.1720 1.4039
Convolution 2684 15239.2305 0.2260 13.2050 5.6778
CopyCPU2CPU 1342 7.6420 0.0030 0.0360 0.0057
log_softmax 1182 36.7860 0.0150 0.0550 0.0311
_backward_copy 1182 60.8850 0.0220 0.0760 0.0515
_backward_Convolution 2364 21876.7051 2.6370 12.5610 9.2541
_mul_scalar 1182 4.2230 0.0020 0.0230 0.0036
_backward_log_softmax 1182 20.0230 0.0090 0.0400 0.0169
mean 3706 71.1050 0.0080 0.0730 0.0192
_backward_mean 1182 12.9390 0.0080 0.1030 0.0109
_backward_FullyConnected 3546 674.4350 0.0280 0.5940 0.1902
WaitForVar 2524 22.5000 0.0060 0.0280 0.0089
broadcast_equal 1342 6.6920 0.0030 0.0110 0.0050
sgd_update 11820 116.5280 0.0020 0.0790 0.0099
_backward_Activation 4728 350.4220 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46208 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4032 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 591 2332.1941 0.3060 73.5850 3.9462
MXNDArrayCreateEx 672 22.9430 0.0310 0.0640 0.0341
MXNDArrayWaitAll 592 12851.7129 5.6130 24.9880 21.7090
MXNDArraySyncCopyToCPU 1264 9959.4307 0.1180 24.1980 7.8793
MXNDArrayGetDType 1264 0.4550 0.0000 0.0010 0.0004
MXNDArrayGetShape 3200 1.4720 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1344 11.7740 0.0070 0.0280 0.0088
MXNDArrayFree 15056 153.4090 0.0000 0.9140 0.0102
MXNet C API Calls 67744 555.1010 487.3580 555.1010 33.8715
MXNet C API Concurrency 135487 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1184 0.5580 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5920 3.0670 0.0000 0.0230 0.0005
MXNDArraySetGradState 5920 2.9950 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1184 0.4910 0.0000 0.0010 0.0004
MXAutogradBackwardEx 592 321.5830 0.5250 0.6810 0.5432
MXNDArrayGetContext 9328 4.0380 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19632 498.5890 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2368 5804.0601 0.7010 3.7500 2.4510
FullyConnected 4032 389.5760 0.0080 0.3430 0.0966
argmax 1344 26.5610 0.0050 0.0480 0.0198
_backward_pick 1184 18.6640 0.0130 0.0260 0.0158
_backward_mul_scalar 1184 5.1220 0.0030 0.0160 0.0043
DeleteVariable 50240 354.4380 0.0010 0.9010 0.0071
Flatten 1344 55.6130 0.0080 0.1120 0.0414
Activation 5376 250.4120 0.0030 1.9440 0.0466
SetValueOp 1184 7.6160 0.0050 0.0180 0.0064
pick 1184 9.2880 0.0060 0.0280 0.0078
Pooling 2688 3773.4890 0.0640 3.1720 1.4038
Convolution 2688 15261.3740 0.2260 13.2050 5.6776
CopyCPU2CPU 1344 7.6520 0.0030 0.0360 0.0057
log_softmax 1184 36.8460 0.0150 0.0550 0.0311
_backward_copy 1184 60.9880 0.0220 0.0760 0.0515
_backward_Convolution 2368 21913.4863 2.6370 12.5610 9.2540
_mul_scalar 1184 4.2300 0.0020 0.0230 0.0036
_backward_log_softmax 1184 20.0560 0.0090 0.0400 0.0169
mean 3712 71.2190 0.0080 0.0730 0.0192
_backward_mean 1184 12.9610 0.0080 0.1030 0.0109
_backward_FullyConnected 3552 675.5100 0.0280 0.5940 0.1902
WaitForVar 2528 22.5350 0.0060 0.0280 0.0089
broadcast_equal 1344 6.7020 0.0030 0.0110 0.0050
sgd_update 11840 116.7310 0.0020 0.0790 0.0099
_backward_Activation 4736 350.9990 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46282 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4038 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 592 2332.6509 0.3060 73.5850 3.9403
MXNDArrayCreateEx 673 22.9760 0.0310 0.0640 0.0341
MXNDArrayWaitAll 593 12873.6201 5.6130 24.9880 21.7093
MXNDArraySyncCopyToCPU 1266 9973.7900 0.1180 24.1980 7.8782
MXNDArrayGetDType 1266 0.4550 0.0000 0.0010 0.0004
MXNDArrayGetShape 3205 1.4750 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1346 11.7910 0.0070 0.0280 0.0088
MXNDArrayFree 15079 153.6560 0.0000 0.9140 0.0102
MXNet C API Calls 67851 555.2080 487.3580 555.2080 33.9250
MXNet C API Concurrency 135701 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1186 0.5580 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5930 3.0710 0.0000 0.0230 0.0005
MXNDArraySetGradState 5930 3.0010 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1186 0.4930 0.0000 0.0010 0.0004
MXAutogradBackwardEx 593 322.1230 0.5250 0.6810 0.5432
MXNDArrayGetContext 9342 4.0430 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19663 499.3770 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2372 5813.8481 0.7010 3.7500 2.4510
FullyConnected 4038 390.1020 0.0080 0.3430 0.0966
argmax 1346 26.6000 0.0050 0.0480 0.0198
_backward_pick 1186 18.6940 0.0130 0.0260 0.0158
_backward_mul_scalar 1186 5.1300 0.0030 0.0160 0.0043
DeleteVariable 50320 354.9920 0.0010 0.9010 0.0071
Flatten 1346 55.6940 0.0080 0.1120 0.0414
Activation 5384 250.7620 0.0030 1.9440 0.0466
SetValueOp 1186 7.6280 0.0050 0.0180 0.0064
pick 1186 9.3030 0.0060 0.0280 0.0078
Pooling 2692 3778.8301 0.0640 3.1720 1.4037
Convolution 2692 15283.5791 0.2260 13.2050 5.6774
CopyCPU2CPU 1346 7.6610 0.0030 0.0360 0.0057
log_softmax 1186 36.9060 0.0150 0.0550 0.0311
_backward_copy 1186 61.1220 0.0220 0.0760 0.0515
_backward_Convolution 2372 21950.5762 2.6370 12.5610 9.2540
_mul_scalar 1186 4.2370 0.0020 0.0230 0.0036
_backward_log_softmax 1186 20.0890 0.0090 0.0400 0.0169
mean 3718 71.3300 0.0080 0.0730 0.0192
_backward_mean 1186 12.9820 0.0080 0.1030 0.0109
_backward_FullyConnected 3558 676.5740 0.0280 0.5940 0.1902
WaitForVar 2532 22.5700 0.0060 0.0280 0.0089
broadcast_equal 1346 6.7110 0.0030 0.0110 0.0050
sgd_update 11860 116.8780 0.0020 0.0790 0.0099
_backward_Activation 4744 351.6140 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46356 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4044 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 593 2336.5759 0.3060 73.5850 3.9403
MXNDArrayCreateEx 674 23.0090 0.0310 0.0640 0.0341
MXNDArrayWaitAll 594 12895.4932 5.6130 24.9880 21.7096
MXNDArraySyncCopyToCPU 1268 9988.1826 0.1180 24.1980 7.8771
MXNDArrayGetDType 1268 0.4560 0.0000 0.0010 0.0004
MXNDArrayGetShape 3210 1.4790 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1348 11.8090 0.0070 0.0280 0.0088
MXNDArrayFree 15102 153.8860 0.0000 0.9140 0.0102
MXNet C API Calls 67958 555.3150 487.3580 555.3150 33.9785
MXNet C API Concurrency 135915 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1188 0.5590 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5940 3.0780 0.0000 0.0230 0.0005
MXNDArraySetGradState 5940 3.0050 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1188 0.4940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 594 322.6880 0.5250 0.6810 0.5432
MXNDArrayGetContext 9356 4.0470 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19694 500.1530 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2376 5823.5479 0.7010 3.7500 2.4510
FullyConnected 4044 390.6550 0.0080 0.3430 0.0966
argmax 1348 26.6380 0.0050 0.0480 0.0198
_backward_pick 1188 18.7260 0.0130 0.0260 0.0158
_backward_mul_scalar 1188 5.1380 0.0030 0.0160 0.0043
DeleteVariable 50400 355.5240 0.0010 0.9010 0.0071
Flatten 1348 55.7760 0.0080 0.1120 0.0414
Activation 5392 251.1180 0.0030 1.9440 0.0466
SetValueOp 1188 7.6400 0.0050 0.0180 0.0064
pick 1188 9.3200 0.0060 0.0280 0.0078
Pooling 2696 3784.2180 0.0640 3.1720 1.4036
Convolution 2696 15305.8574 0.2260 13.2050 5.6772
CopyCPU2CPU 1348 7.6700 0.0030 0.0360 0.0057
log_softmax 1188 36.9660 0.0150 0.0550 0.0311
_backward_copy 1188 61.2260 0.0220 0.0760 0.0515
_backward_Convolution 2376 21987.6738 2.6370 12.5610 9.2541
_mul_scalar 1188 4.2440 0.0020 0.0230 0.0036
_backward_log_softmax 1188 20.1210 0.0090 0.0400 0.0169
mean 3724 71.4460 0.0080 0.0730 0.0192
_backward_mean 1188 13.0030 0.0080 0.1030 0.0109
_backward_FullyConnected 3564 677.6380 0.0280 0.5940 0.1901
WaitForVar 2536 22.6050 0.0060 0.0280 0.0089
broadcast_equal 1348 6.7200 0.0030 0.0110 0.0050
sgd_update 11880 117.0840 0.0020 0.0790 0.0099
_backward_Activation 4752 352.1450 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46430 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4050 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 594 2340.4919 0.3060 73.5850 3.9402
MXNDArrayCreateEx 675 23.0430 0.0310 0.0640 0.0341
MXNDArrayWaitAll 595 12917.1729 5.6130 24.9880 21.7095
MXNDArraySyncCopyToCPU 1270 10002.5557 0.1180 24.1980 7.8760
MXNDArrayGetDType 1270 0.4560 0.0000 0.0010 0.0004
MXNDArrayGetShape 3215 1.4810 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1350 11.8270 0.0070 0.0280 0.0088
MXNDArrayFree 15125 154.1140 0.0000 0.9140 0.0102
MXNet C API Calls 68065 555.4220 487.3580 555.4220 34.0320
MXNet C API Concurrency 136129 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1190 0.5590 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5950 3.0850 0.0000 0.0230 0.0005
MXNDArraySetGradState 5950 3.0100 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1190 0.4940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 595 323.2470 0.5250 0.6810 0.5433
MXNDArrayGetContext 9370 4.0540 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19725 500.9750 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2380 5833.2339 0.7010 3.7500 2.4509
FullyConnected 4050 391.2470 0.0080 0.3430 0.0966
argmax 1350 26.6760 0.0050 0.0480 0.0198
_backward_pick 1190 18.7610 0.0130 0.0260 0.0158
_backward_mul_scalar 1190 5.1470 0.0030 0.0160 0.0043
DeleteVariable 50480 356.0600 0.0010 0.9010 0.0071
Flatten 1350 55.8580 0.0080 0.1120 0.0414
Activation 5400 251.4890 0.0030 1.9440 0.0466
SetValueOp 1190 7.6530 0.0050 0.0180 0.0064
pick 1190 9.3360 0.0060 0.0280 0.0078
Pooling 2700 3789.5669 0.0640 3.1720 1.4035
Convolution 2700 15328.0664 0.2260 13.2050 5.6771
CopyCPU2CPU 1350 7.6800 0.0030 0.0360 0.0057
log_softmax 1190 37.0380 0.0150 0.0550 0.0311
_backward_copy 1190 61.3270 0.0220 0.0760 0.0515
_backward_Convolution 2380 22024.4199 2.6370 12.5610 9.2540
_mul_scalar 1190 4.2510 0.0020 0.0230 0.0036
_backward_log_softmax 1190 20.1540 0.0090 0.0400 0.0169
mean 3730 71.5600 0.0080 0.0730 0.0192
_backward_mean 1190 13.0240 0.0080 0.1030 0.0109
_backward_FullyConnected 3570 678.8240 0.0280 0.5940 0.1901
WaitForVar 2540 22.6390 0.0060 0.0280 0.0089
broadcast_equal 1350 6.7300 0.0030 0.0110 0.0050
sgd_update 11900 117.2560 0.0020 0.0790 0.0099
_backward_Activation 4760 352.7720 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46504 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4056 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 595 2344.4331 0.3060 73.5850 3.9402
MXNDArrayCreateEx 676 23.0780 0.0310 0.0640 0.0341
MXNDArrayWaitAll 596 12938.9023 5.6130 24.9880 21.7096
MXNDArraySyncCopyToCPU 1272 10016.8701 0.1180 24.1980 7.8749
MXNDArrayGetDType 1272 0.4560 0.0000 0.0010 0.0004
MXNDArrayGetShape 3220 1.4830 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1352 11.8450 0.0070 0.0280 0.0088
MXNDArrayFree 15148 154.3580 0.0000 0.9140 0.0102
MXNet C API Calls 68172 555.5290 487.3580 555.5290 34.0855
MXNet C API Concurrency 136343 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1192 0.5590 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5960 3.0900 0.0000 0.0230 0.0005
MXNDArraySetGradState 5960 3.0150 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1192 0.4940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 596 323.7940 0.5250 0.6810 0.5433
MXNDArrayGetContext 9384 4.0620 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19756 501.7590 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2384 5842.8760 0.7010 3.7500 2.4509
FullyConnected 4056 391.8120 0.0080 0.3430 0.0966
argmax 1352 26.7150 0.0050 0.0480 0.0198
_backward_pick 1192 18.7940 0.0130 0.0260 0.0158
_backward_mul_scalar 1192 5.1540 0.0030 0.0160 0.0043
DeleteVariable 50560 356.6310 0.0010 0.9010 0.0071
Flatten 1352 55.9450 0.0080 0.1120 0.0414
Activation 5408 251.8480 0.0030 1.9440 0.0466
SetValueOp 1192 7.6640 0.0050 0.0180 0.0064
pick 1192 9.3510 0.0060 0.0280 0.0078
Pooling 2704 3794.9250 0.0640 3.1720 1.4034
Convolution 2704 15350.2031 0.2260 13.2050 5.6769
CopyCPU2CPU 1352 7.6900 0.0030 0.0360 0.0057
log_softmax 1192 37.0980 0.0150 0.0550 0.0311
_backward_copy 1192 61.4300 0.0220 0.0760 0.0515
_backward_Convolution 2384 22061.2891 2.6370 12.5610 9.2539
_mul_scalar 1192 4.2580 0.0020 0.0230 0.0036
_backward_log_softmax 1192 20.1850 0.0090 0.0400 0.0169
mean 3736 71.6750 0.0080 0.0730 0.0192
_backward_mean 1192 13.0450 0.0080 0.1030 0.0109
_backward_FullyConnected 3576 679.8980 0.0280 0.5940 0.1901
WaitForVar 2544 22.6720 0.0060 0.0280 0.0089
broadcast_equal 1352 6.7400 0.0030 0.0110 0.0050
sgd_update 11920 117.4580 0.0020 0.0790 0.0099
_backward_Activation 4768 353.3240 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46578 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4062 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 596 2348.3870 0.3060 73.5850 3.9402
MXNDArrayCreateEx 677 23.1130 0.0310 0.0640 0.0341
MXNDArrayWaitAll 597 12960.7764 5.6130 24.9880 21.7098
MXNDArraySyncCopyToCPU 1274 10031.0654 0.1180 24.1980 7.8737
MXNDArrayGetDType 1274 0.4580 0.0000 0.0010 0.0004
MXNDArrayGetShape 3225 1.4870 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1354 11.8630 0.0070 0.0280 0.0088
MXNDArrayFree 15171 154.6030 0.0000 0.9140 0.0102
MXNet C API Calls 68279 555.6360 487.3580 555.6360 34.1390
MXNet C API Concurrency 136557 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1194 0.5600 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5970 3.0950 0.0000 0.0230 0.0005
MXNDArraySetGradState 5970 3.0240 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1194 0.4950 0.0000 0.0010 0.0004
MXAutogradBackwardEx 597 324.3700 0.5250 0.6810 0.5433
MXNDArrayGetContext 9398 4.0670 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19787 502.5780 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2388 5852.5742 0.7010 3.7500 2.4508
FullyConnected 4062 392.3480 0.0080 0.3430 0.0966
argmax 1354 26.7520 0.0050 0.0480 0.0198
_backward_pick 1194 18.8250 0.0130 0.0260 0.0158
_backward_mul_scalar 1194 5.1620 0.0030 0.0160 0.0043
DeleteVariable 50640 357.1750 0.0010 0.9010 0.0071
Flatten 1354 56.0380 0.0080 0.1120 0.0414
Activation 5416 252.2080 0.0030 1.9440 0.0466
SetValueOp 1194 7.6760 0.0050 0.0180 0.0064
pick 1194 9.3680 0.0060 0.0280 0.0078
Pooling 2708 3800.3311 0.0640 3.1720 1.4034
Convolution 2708 15372.3975 0.2260 13.2050 5.6767
CopyCPU2CPU 1354 7.7000 0.0030 0.0360 0.0057
log_softmax 1194 37.1580 0.0150 0.0550 0.0311
_backward_copy 1194 61.5330 0.0220 0.0760 0.0515
_backward_Convolution 2388 22098.4180 2.6370 12.5610 9.2539
_mul_scalar 1194 4.2650 0.0020 0.0230 0.0036
_backward_log_softmax 1194 20.2170 0.0090 0.0400 0.0169
mean 3742 71.7920 0.0080 0.0730 0.0192
_backward_mean 1194 13.0680 0.0080 0.1030 0.0109
_backward_FullyConnected 3582 680.9710 0.0280 0.5940 0.1901
WaitForVar 2548 22.7060 0.0060 0.0280 0.0089
broadcast_equal 1354 6.7490 0.0030 0.0110 0.0050
sgd_update 11940 117.6060 0.0020 0.0790 0.0098
_backward_Activation 4776 353.9440 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46652 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4068 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 597 2352.3401 0.3060 73.5850 3.9403
MXNDArrayCreateEx 678 23.1490 0.0310 0.0640 0.0341
MXNDArrayWaitAll 598 12982.6904 5.6130 24.9880 21.7102
MXNDArraySyncCopyToCPU 1276 10045.4346 0.1180 24.1980 7.8726
MXNDArrayGetDType 1276 0.4600 0.0000 0.0010 0.0004
MXNDArrayGetShape 3230 1.4910 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1356 11.8810 0.0070 0.0280 0.0088
MXNDArrayFree 15194 154.8520 0.0000 0.9140 0.0102
MXNet C API Calls 68386 555.7430 487.3580 555.7430 34.1925
MXNet C API Concurrency 136771 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1196 0.5610 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5980 3.0980 0.0000 0.0230 0.0005
MXNDArraySetGradState 5980 3.0260 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1196 0.4950 0.0000 0.0010 0.0004
MXAutogradBackwardEx 598 324.9200 0.5250 0.6810 0.5433
MXNDArrayGetContext 9412 4.0730 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19818 503.3940 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2392 5862.2109 0.7010 3.7500 2.4508
FullyConnected 4068 392.8920 0.0080 0.3430 0.0966
argmax 1356 26.7900 0.0050 0.0480 0.0198
_backward_pick 1196 18.8570 0.0130 0.0260 0.0158
_backward_mul_scalar 1196 5.1700 0.0030 0.0160 0.0043
DeleteVariable 50720 357.7670 0.0010 0.9010 0.0071
Flatten 1356 56.1320 0.0080 0.1120 0.0414
Activation 5424 252.5690 0.0030 1.9440 0.0466
SetValueOp 1196 7.6890 0.0050 0.0180 0.0064
pick 1196 9.3830 0.0060 0.0280 0.0078
Pooling 2712 3805.6790 0.0640 3.1720 1.4033
Convolution 2712 15394.7207 0.2260 13.2050 5.6765
CopyCPU2CPU 1356 7.7090 0.0030 0.0360 0.0057
log_softmax 1196 37.2180 0.0150 0.0550 0.0311
_backward_copy 1196 61.6360 0.0220 0.0760 0.0515
_backward_Convolution 2392 22135.5918 2.6370 12.5610 9.2540
_mul_scalar 1196 4.2720 0.0020 0.0230 0.0036
_backward_log_softmax 1196 20.2490 0.0090 0.0400 0.0169
mean 3748 71.9150 0.0080 0.0730 0.0192
_backward_mean 1196 13.0900 0.0080 0.1030 0.0109
_backward_FullyConnected 3588 682.0990 0.0280 0.5940 0.1901
WaitForVar 2552 22.7400 0.0060 0.0280 0.0089
broadcast_equal 1356 6.7600 0.0030 0.0110 0.0050
sgd_update 11960 117.7550 0.0020 0.0790 0.0098
_backward_Activation 4784 354.5410 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46726 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4074 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 598 2356.2710 0.3060 73.5850 3.9403
MXNDArrayCreateEx 679 23.1840 0.0310 0.0640 0.0341
MXNDArrayWaitAll 599 13004.4287 5.6130 24.9880 21.7102
MXNDArraySyncCopyToCPU 1278 10059.9346 0.1180 24.1980 7.8716
MXNDArrayGetDType 1278 0.4610 0.0000 0.0010 0.0004
MXNDArrayGetShape 3235 1.4930 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1358 11.8990 0.0070 0.0280 0.0088
MXNDArrayFree 15217 155.0770 0.0000 0.9140 0.0102
MXNet C API Calls 68493 555.8500 487.3580 555.8500 34.2460
MXNet C API Concurrency 136985 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1198 0.5620 0.0000 0.0040 0.0005
MXNDArrayGetGradState 5990 3.0990 0.0000 0.0230 0.0005
MXNDArraySetGradState 5990 3.0300 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1198 0.4950 0.0000 0.0010 0.0004
MXAutogradBackwardEx 599 325.4640 0.5250 0.6810 0.5433
MXNDArrayGetContext 9426 4.0780 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19849 504.1790 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2396 5871.9438 0.7010 3.7500 2.4507
FullyConnected 4074 393.4930 0.0080 0.3430 0.0966
argmax 1358 26.8330 0.0050 0.0480 0.0198
_backward_pick 1198 18.8880 0.0130 0.0260 0.0158
_backward_mul_scalar 1198 5.1780 0.0030 0.0160 0.0043
DeleteVariable 50800 358.3040 0.0010 0.9010 0.0071
Flatten 1358 56.2130 0.0080 0.1120 0.0414
Activation 5432 252.9570 0.0030 1.9440 0.0466
SetValueOp 1198 7.7020 0.0050 0.0180 0.0064
pick 1198 9.3990 0.0060 0.0280 0.0078
Pooling 2716 3811.0371 0.0640 3.1720 1.4032
Convolution 2716 15416.9141 0.2260 13.2050 5.6763
CopyCPU2CPU 1358 7.7190 0.0030 0.0360 0.0057
log_softmax 1198 37.2780 0.0150 0.0550 0.0311
_backward_copy 1198 61.7370 0.0220 0.0760 0.0515
_backward_Convolution 2396 22172.4160 2.6370 12.5610 9.2539
_mul_scalar 1198 4.2790 0.0020 0.0230 0.0036
_backward_log_softmax 1198 20.2920 0.0090 0.0400 0.0169
mean 3754 72.0160 0.0080 0.0730 0.0192
_backward_mean 1198 13.1120 0.0080 0.1030 0.0109
_backward_FullyConnected 3594 683.2830 0.0280 0.5940 0.1901
WaitForVar 2556 22.7740 0.0060 0.0280 0.0089
broadcast_equal 1358 6.7690 0.0030 0.0110 0.0050
sgd_update 11980 117.9950 0.0020 0.0790 0.0098
_backward_Activation 4792 355.1000 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46800 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4080 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 599 2360.1931 0.3060 73.5850 3.9402
MXNDArrayCreateEx 680 23.2190 0.0310 0.0640 0.0341
MXNDArrayWaitAll 600 13026.1094 5.6130 24.9880 21.7102
MXNDArraySyncCopyToCPU 1280 10074.3018 0.1180 24.1980 7.8705
MXNDArrayGetDType 1280 0.4620 0.0000 0.0010 0.0004
MXNDArrayGetShape 3240 1.4940 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1360 11.9170 0.0070 0.0280 0.0088
MXNDArrayFree 15240 155.3100 0.0000 0.9140 0.0102
MXNet C API Calls 68600 555.9570 487.3580 555.9570 34.2995
MXNet C API Concurrency 137199 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1200 0.5640 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6000 3.1030 0.0000 0.0230 0.0005
MXNDArraySetGradState 6000 3.0340 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1200 0.4960 0.0000 0.0010 0.0004
MXAutogradBackwardEx 600 326.0060 0.5250 0.6810 0.5433
MXNDArrayGetContext 9440 4.0880 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19880 504.9580 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2400 5881.5620 0.7010 3.7500 2.4507
FullyConnected 4080 394.0450 0.0080 0.3430 0.0966
argmax 1360 26.8710 0.0050 0.0480 0.0198
_backward_pick 1200 18.9180 0.0130 0.0260 0.0158
_backward_mul_scalar 1200 5.1860 0.0030 0.0160 0.0043
DeleteVariable 50880 358.8600 0.0010 0.9010 0.0071
Flatten 1360 56.2970 0.0080 0.1120 0.0414
Activation 5440 253.3110 0.0030 1.9440 0.0466
SetValueOp 1200 7.7140 0.0050 0.0180 0.0064
pick 1200 9.4150 0.0060 0.0280 0.0078
Pooling 2720 3816.3721 0.0640 3.1720 1.4031
Convolution 2720 15439.0518 0.2260 13.2050 5.6761
CopyCPU2CPU 1360 7.7290 0.0030 0.0360 0.0057
log_softmax 1200 37.3370 0.0150 0.0550 0.0311
_backward_copy 1200 61.8410 0.0220 0.0760 0.0515
_backward_Convolution 2400 22209.2285 2.6370 12.5610 9.2538
_mul_scalar 1200 4.2860 0.0020 0.0230 0.0036
_backward_log_softmax 1200 20.3250 0.0090 0.0400 0.0169
mean 3760 72.1310 0.0080 0.0730 0.0192
_backward_mean 1200 13.1330 0.0080 0.1030 0.0109
_backward_FullyConnected 3600 684.3600 0.0280 0.5940 0.1901
WaitForVar 2560 22.8070 0.0060 0.0280 0.0089
broadcast_equal 1360 6.7800 0.0030 0.0110 0.0050
sgd_update 12000 118.1990 0.0020 0.0790 0.0098
_backward_Activation 4800 355.6790 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46874 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4086 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 600 2364.1331 0.3060 73.5850 3.9402
MXNDArrayCreateEx 681 23.2530 0.0310 0.0640 0.0341
MXNDArrayWaitAll 601 13047.9707 5.6130 24.9880 21.7104
MXNDArraySyncCopyToCPU 1282 10088.4600 0.1180 24.1980 7.8693
MXNDArrayGetDType 1282 0.4630 0.0000 0.0010 0.0004
MXNDArrayGetShape 3245 1.4990 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1362 11.9350 0.0070 0.0280 0.0088
MXNDArrayFree 15263 155.5470 0.0000 0.9140 0.0102
MXNet C API Calls 68707 556.0640 487.3580 556.0640 34.3530
MXNet C API Concurrency 137413 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1202 0.5650 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6010 3.1080 0.0000 0.0230 0.0005
MXNDArraySetGradState 6010 3.0390 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1202 0.4970 0.0000 0.0010 0.0004
MXAutogradBackwardEx 601 326.5960 0.5250 0.6810 0.5434
MXNDArrayGetContext 9454 4.0940 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19911 505.7680 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2404 5891.2290 0.7010 3.7500 2.4506
FullyConnected 4086 394.5690 0.0080 0.3430 0.0966
argmax 1362 26.9090 0.0050 0.0480 0.0198
_backward_pick 1202 18.9500 0.0130 0.0260 0.0158
_backward_mul_scalar 1202 5.1950 0.0030 0.0160 0.0043
DeleteVariable 50960 359.4180 0.0010 0.9010 0.0071
Flatten 1362 56.3760 0.0080 0.1120 0.0414
Activation 5448 253.6640 0.0030 1.9440 0.0466
SetValueOp 1202 7.7260 0.0050 0.0180 0.0064
pick 1202 9.4310 0.0060 0.0280 0.0078
Pooling 2724 3821.7429 0.0640 3.1720 1.4030
Convolution 2724 15461.2490 0.2260 13.2050 5.6759
CopyCPU2CPU 1362 7.7390 0.0030 0.0360 0.0057
log_softmax 1202 37.3960 0.0150 0.0550 0.0311
_backward_copy 1202 61.9440 0.0220 0.0760 0.0515
_backward_Convolution 2404 22246.3281 2.6370 12.5610 9.2539
_mul_scalar 1202 4.2920 0.0020 0.0230 0.0036
_backward_log_softmax 1202 20.3580 0.0090 0.0400 0.0169
mean 3766 72.2480 0.0080 0.0730 0.0192
_backward_mean 1202 13.1550 0.0080 0.1030 0.0109
_backward_FullyConnected 3606 685.4320 0.0280 0.5940 0.1901
WaitForVar 2564 22.8440 0.0060 0.0280 0.0089
broadcast_equal 1362 6.7900 0.0030 0.0110 0.0050
sgd_update 12020 118.4040 0.0020 0.0790 0.0099
_backward_Activation 4808 356.2100 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 46948 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4092 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 601 2368.0659 0.3060 73.5850 3.9402
MXNDArrayCreateEx 682 23.2880 0.0310 0.0640 0.0341
MXNDArrayWaitAll 602 13069.7910 5.6130 24.9880 21.7106
MXNDArraySyncCopyToCPU 1284 10102.8662 0.1180 24.1980 7.8683
MXNDArrayGetDType 1284 0.4630 0.0000 0.0010 0.0004
MXNDArrayGetShape 3250 1.5030 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1364 11.9510 0.0070 0.0280 0.0088
MXNDArrayFree 15286 155.7790 0.0000 0.9140 0.0102
MXNet C API Calls 68814 556.1710 487.3580 556.1710 34.4065
MXNet C API Concurrency 137627 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1204 0.5660 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6020 3.1140 0.0000 0.0230 0.0005
MXNDArraySetGradState 6020 3.0430 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1204 0.4970 0.0000 0.0010 0.0004
MXAutogradBackwardEx 602 327.1370 0.5250 0.6810 0.5434
MXNDArrayGetContext 9468 4.0980 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19942 506.5510 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2408 5900.8818 0.7010 3.7500 2.4505
FullyConnected 4092 395.1220 0.0080 0.3430 0.0966
argmax 1364 26.9480 0.0050 0.0480 0.0198
_backward_pick 1204 18.9820 0.0130 0.0260 0.0158
_backward_mul_scalar 1204 5.2020 0.0030 0.0160 0.0043
DeleteVariable 51040 359.9620 0.0010 0.9010 0.0071
Flatten 1364 56.4610 0.0080 0.1120 0.0414
Activation 5456 254.0190 0.0030 1.9440 0.0466
SetValueOp 1204 7.7390 0.0050 0.0180 0.0064
pick 1204 9.4460 0.0060 0.0280 0.0078
Pooling 2728 3827.1001 0.0640 3.1720 1.4029
Convolution 2728 15483.4883 0.2260 13.2050 5.6758
CopyCPU2CPU 1364 7.7490 0.0030 0.0360 0.0057
log_softmax 1204 37.4560 0.0150 0.0550 0.0311
_backward_copy 1204 62.0460 0.0220 0.0760 0.0515
_backward_Convolution 2408 22283.3574 2.6370 12.5610 9.2539
_mul_scalar 1204 4.2990 0.0020 0.0230 0.0036
_backward_log_softmax 1204 20.3910 0.0090 0.0400 0.0169
mean 3772 72.3640 0.0080 0.0730 0.0192
_backward_mean 1204 13.1750 0.0080 0.1030 0.0109
_backward_FullyConnected 3612 686.5040 0.0280 0.5940 0.1901
WaitForVar 2568 22.8790 0.0060 0.0280 0.0089
broadcast_equal 1364 6.8000 0.0030 0.0110 0.0050
sgd_update 12040 118.6080 0.0020 0.0790 0.0099
_backward_Activation 4816 356.7600 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47022 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4098 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 602 2371.9839 0.3060 73.5850 3.9402
MXNDArrayCreateEx 683 23.3220 0.0310 0.0640 0.0341
MXNDArrayWaitAll 603 13091.4932 5.6130 24.9880 21.7106
MXNDArraySyncCopyToCPU 1286 10117.3232 0.1180 24.1980 7.8673
MXNDArrayGetDType 1286 0.4640 0.0000 0.0010 0.0004
MXNDArrayGetShape 3255 1.5060 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1366 11.9680 0.0070 0.0280 0.0088
MXNDArrayFree 15309 156.0130 0.0000 0.9140 0.0102
MXNet C API Calls 68921 556.2780 487.3580 556.2780 34.4600
MXNet C API Concurrency 137841 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1206 0.5680 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6030 3.1210 0.0000 0.0230 0.0005
MXNDArraySetGradState 6030 3.0490 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1206 0.4980 0.0000 0.0010 0.0004
MXAutogradBackwardEx 603 327.6790 0.5250 0.6810 0.5434
MXNDArrayGetContext 9482 4.1090 0.0000 0.0070 0.0004
MXImperativeInvokeEx 19973 507.3340 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2412 5910.5542 0.7010 3.7500 2.4505
FullyConnected 4098 395.7640 0.0080 0.3430 0.0966
argmax 1366 26.9910 0.0050 0.0480 0.0198
_backward_pick 1206 19.0130 0.0130 0.0260 0.0158
_backward_mul_scalar 1206 5.2110 0.0030 0.0160 0.0043
DeleteVariable 51120 360.5100 0.0010 0.9010 0.0071
Flatten 1366 56.5410 0.0080 0.1120 0.0414
Activation 5464 254.3870 0.0030 1.9440 0.0466
SetValueOp 1206 7.7500 0.0050 0.0180 0.0064
pick 1206 9.4630 0.0060 0.0280 0.0078
Pooling 2732 3832.4600 0.0640 3.1720 1.4028
Convolution 2732 15505.6758 0.2260 13.2050 5.6756
CopyCPU2CPU 1366 7.7590 0.0030 0.0360 0.0057
log_softmax 1206 37.5280 0.0150 0.0550 0.0311
_backward_copy 1206 62.1490 0.0220 0.0760 0.0515
_backward_Convolution 2412 22320.1191 2.6370 12.5610 9.2538
_mul_scalar 1206 4.3060 0.0020 0.0230 0.0036
_backward_log_softmax 1206 20.4220 0.0090 0.0400 0.0169
mean 3778 72.4700 0.0080 0.0730 0.0192
_backward_mean 1206 13.1970 0.0080 0.1030 0.0109
_backward_FullyConnected 3618 687.6900 0.0280 0.5940 0.1901
WaitForVar 2572 22.9150 0.0060 0.0280 0.0089
broadcast_equal 1366 6.8100 0.0030 0.0110 0.0050
sgd_update 12060 118.8220 0.0020 0.0790 0.0099
_backward_Activation 4824 357.3220 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47096 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4104 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 603 2375.8999 0.3060 73.5850 3.9401
MXNDArrayCreateEx 684 23.3570 0.0310 0.0640 0.0341
MXNDArrayWaitAll 604 13113.2051 5.6130 24.9880 21.7106
MXNDArraySyncCopyToCPU 1288 10131.6641 0.1180 24.1980 7.8662
MXNDArrayGetDType 1288 0.4650 0.0000 0.0010 0.0004
MXNDArrayGetShape 3260 1.5090 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1368 11.9860 0.0070 0.0280 0.0088
MXNDArrayFree 15332 156.2370 0.0000 0.9140 0.0102
MXNet C API Calls 69028 556.3850 487.3580 556.3850 34.5135
MXNet C API Concurrency 138055 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1208 0.5690 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6040 3.1280 0.0000 0.0230 0.0005
MXNDArraySetGradState 6040 3.0550 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1208 0.4990 0.0000 0.0010 0.0004
MXAutogradBackwardEx 604 328.2220 0.5250 0.6810 0.5434
MXNDArrayGetContext 9496 4.1150 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20004 508.1190 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2416 5920.2598 0.7010 3.7500 2.4504
FullyConnected 4104 396.3060 0.0080 0.3430 0.0966
argmax 1368 27.0290 0.0050 0.0480 0.0198
_backward_pick 1208 19.0440 0.0130 0.0260 0.0158
_backward_mul_scalar 1208 5.2200 0.0030 0.0160 0.0043
DeleteVariable 51200 361.0380 0.0010 0.9010 0.0071
Flatten 1368 56.6220 0.0080 0.1120 0.0414
Activation 5472 254.7380 0.0030 1.9440 0.0466
SetValueOp 1208 7.7620 0.0050 0.0180 0.0064
pick 1208 9.4780 0.0060 0.0280 0.0078
Pooling 2736 3837.8621 0.0640 3.1720 1.4027
Convolution 2736 15527.7998 0.2260 13.2050 5.6754
CopyCPU2CPU 1368 7.7690 0.0030 0.0360 0.0057
log_softmax 1208 37.5870 0.0150 0.0550 0.0311
_backward_copy 1208 62.2500 0.0220 0.0760 0.0515
_backward_Convolution 2416 22356.9238 2.6370 12.5610 9.2537
_mul_scalar 1208 4.3130 0.0020 0.0230 0.0036
_backward_log_softmax 1208 20.4540 0.0090 0.0400 0.0169
mean 3784 72.5800 0.0080 0.0730 0.0192
_backward_mean 1208 13.2180 0.0080 0.1030 0.0109
_backward_FullyConnected 3624 688.7620 0.0280 0.5940 0.1901
WaitForVar 2576 22.9500 0.0060 0.0280 0.0089
broadcast_equal 1368 6.8190 0.0030 0.0110 0.0050
sgd_update 12080 119.0220 0.0020 0.0790 0.0099
_backward_Activation 4832 357.8750 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47170 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4110 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 604 2379.8159 0.3060 73.5850 3.9401
MXNDArrayCreateEx 685 23.3920 0.0310 0.0640 0.0341
MXNDArrayWaitAll 605 13135.0000 5.6130 24.9880 21.7107
MXNDArraySyncCopyToCPU 1290 10146.0420 0.1180 24.1980 7.8651
MXNDArrayGetDType 1290 0.4670 0.0000 0.0010 0.0004
MXNDArrayGetShape 3265 1.5100 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1370 12.0040 0.0070 0.0280 0.0088
MXNDArrayFree 15355 156.4750 0.0000 0.9140 0.0102
MXNet C API Calls 69135 556.4920 487.3580 556.4920 34.5670
MXNet C API Concurrency 138269 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1210 0.5700 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6050 3.1310 0.0000 0.0230 0.0005
MXNDArraySetGradState 6050 3.0600 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1210 0.4990 0.0000 0.0010 0.0004
MXAutogradBackwardEx 605 328.7630 0.5250 0.6810 0.5434
MXNDArrayGetContext 9510 4.1230 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20035 508.8970 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2420 5929.8271 0.7010 3.7500 2.4503
FullyConnected 4110 396.8510 0.0080 0.3430 0.0966
argmax 1370 27.0660 0.0050 0.0480 0.0198
_backward_pick 1210 19.0770 0.0130 0.0260 0.0158
_backward_mul_scalar 1210 5.2290 0.0030 0.0160 0.0043
DeleteVariable 51280 361.5820 0.0010 0.9010 0.0071
Flatten 1370 56.7040 0.0080 0.1120 0.0414
Activation 5480 255.0920 0.0030 1.9440 0.0465
SetValueOp 1210 7.7740 0.0050 0.0180 0.0064
pick 1210 9.4950 0.0060 0.0280 0.0078
Pooling 2740 3843.2190 0.0640 3.1720 1.4026
Convolution 2740 15550.0391 0.2260 13.2050 5.6752
CopyCPU2CPU 1370 7.7790 0.0030 0.0360 0.0057
log_softmax 1210 37.6470 0.0150 0.0550 0.0311
_backward_copy 1210 62.3930 0.0220 0.0760 0.0516
_backward_Convolution 2420 22393.9473 2.6370 12.5610 9.2537
_mul_scalar 1210 4.3200 0.0020 0.0230 0.0036
_backward_log_softmax 1210 20.4860 0.0090 0.0400 0.0169
mean 3790 72.6970 0.0080 0.0730 0.0192
_backward_mean 1210 13.2390 0.0080 0.1030 0.0109
_backward_FullyConnected 3630 689.8310 0.0280 0.5940 0.1900
WaitForVar 2580 22.9860 0.0060 0.0280 0.0089
broadcast_equal 1370 6.8290 0.0030 0.0110 0.0050
sgd_update 12100 119.2240 0.0020 0.0790 0.0099
_backward_Activation 4840 358.4030 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47244 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4116 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 605 2383.7329 0.3060 73.5850 3.9401
MXNDArrayCreateEx 686 23.4270 0.0310 0.0640 0.0342
MXNDArrayWaitAll 606 13156.8330 5.6130 24.9880 21.7109
MXNDArraySyncCopyToCPU 1292 10160.3965 0.1180 24.1980 7.8641
MXNDArrayGetDType 1292 0.4680 0.0000 0.0010 0.0004
MXNDArrayGetShape 3270 1.5120 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1372 12.0210 0.0070 0.0280 0.0088
MXNDArrayFree 15378 156.6990 0.0000 0.9140 0.0102
MXNet C API Calls 69242 556.5990 487.3580 556.5990 34.6205
MXNet C API Concurrency 138483 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1212 0.5700 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6060 3.1380 0.0000 0.0230 0.0005
MXNDArraySetGradState 6060 3.0650 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1212 0.5010 0.0000 0.0010 0.0004
MXAutogradBackwardEx 606 329.3270 0.5250 0.6810 0.5434
MXNDArrayGetContext 9524 4.1300 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20066 509.6860 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2424 5939.4482 0.7010 3.7500 2.4503
FullyConnected 4116 397.3740 0.0080 0.3430 0.0965
argmax 1372 27.1040 0.0050 0.0480 0.0198
_backward_pick 1212 19.1080 0.0130 0.0260 0.0158
_backward_mul_scalar 1212 5.2370 0.0030 0.0160 0.0043
DeleteVariable 51360 362.1100 0.0010 0.9010 0.0071
Flatten 1372 56.7830 0.0080 0.1120 0.0414
Activation 5488 255.4450 0.0030 1.9440 0.0465
SetValueOp 1212 7.7860 0.0050 0.0180 0.0064
pick 1212 9.5100 0.0060 0.0280 0.0078
Pooling 2744 3848.6030 0.0640 3.1720 1.4026
Convolution 2744 15572.2939 0.2260 13.2050 5.6750
CopyCPU2CPU 1372 7.7890 0.0030 0.0360 0.0057
log_softmax 1212 37.7060 0.0150 0.0550 0.0311
_backward_copy 1212 62.4970 0.0220 0.0760 0.0516
_backward_Convolution 2424 22431.0039 2.6370 12.5610 9.2537
_mul_scalar 1212 4.3260 0.0020 0.0230 0.0036
_backward_log_softmax 1212 20.5190 0.0090 0.0400 0.0169
mean 3796 72.8140 0.0080 0.0730 0.0192
_backward_mean 1212 13.2600 0.0080 0.1030 0.0109
_backward_FullyConnected 3636 690.8990 0.0280 0.5940 0.1900
WaitForVar 2584 23.0210 0.0060 0.0280 0.0089
broadcast_equal 1372 6.8380 0.0030 0.0110 0.0050
sgd_update 12120 119.4250 0.0020 0.0790 0.0099
_backward_Activation 4848 358.9700 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47318 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4122 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 606 2387.6599 0.3060 73.5850 3.9400
MXNDArrayCreateEx 687 23.4610 0.0310 0.0640 0.0341
MXNDArrayWaitAll 607 13178.5840 5.6130 24.9880 21.7110
MXNDArraySyncCopyToCPU 1294 10174.9160 0.1180 24.1980 7.8632
MXNDArrayGetDType 1294 0.4680 0.0000 0.0010 0.0004
MXNDArrayGetShape 3275 1.5150 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1374 12.0380 0.0070 0.0280 0.0088
MXNDArrayFree 15401 156.9360 0.0000 0.9140 0.0102
MXNet C API Calls 69349 556.7060 487.3580 556.7060 34.6740
MXNet C API Concurrency 138697 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1214 0.5700 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6070 3.1410 0.0000 0.0230 0.0005
MXNDArraySetGradState 6070 3.0670 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1214 0.5010 0.0000 0.0010 0.0004
MXAutogradBackwardEx 607 329.8660 0.5250 0.6810 0.5434
MXNDArrayGetContext 9538 4.1370 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20097 510.4690 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2428 5949.2388 0.7010 3.7500 2.4503
FullyConnected 4122 397.9910 0.0080 0.3430 0.0966
argmax 1374 27.1440 0.0050 0.0480 0.0198
_backward_pick 1214 19.1400 0.0130 0.0260 0.0158
_backward_mul_scalar 1214 5.2450 0.0030 0.0160 0.0043
DeleteVariable 51440 362.6760 0.0010 0.9010 0.0071
Flatten 1374 56.8680 0.0080 0.1120 0.0414
Activation 5496 255.8200 0.0030 1.9440 0.0465
SetValueOp 1214 7.7980 0.0050 0.0180 0.0064
pick 1214 9.5280 0.0060 0.0280 0.0078
Pooling 2748 3853.9541 0.0640 3.1720 1.4025
Convolution 2748 15594.4668 0.2260 13.2050 5.6748
CopyCPU2CPU 1374 7.7980 0.0030 0.0360 0.0057
log_softmax 1214 37.7650 0.0150 0.0550 0.0311
_backward_copy 1214 62.5990 0.0220 0.0760 0.0516
_backward_Convolution 2428 22467.7383 2.6370 12.5610 9.2536
_mul_scalar 1214 4.3330 0.0020 0.0230 0.0036
_backward_log_softmax 1214 20.5620 0.0090 0.0400 0.0169
mean 3802 72.9160 0.0080 0.0730 0.0192
_backward_mean 1214 13.2810 0.0080 0.1030 0.0109
_backward_FullyConnected 3642 692.0900 0.0280 0.5940 0.1900
WaitForVar 2588 23.0590 0.0060 0.0280 0.0089
broadcast_equal 1374 6.8480 0.0030 0.0110 0.0050
sgd_update 12140 119.6440 0.0020 0.0790 0.0099
_backward_Activation 4856 359.5690 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47392 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4128 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 607 2391.5730 0.3060 73.5850 3.9400
MXNDArrayCreateEx 688 23.4950 0.0310 0.0640 0.0341
MXNDArrayWaitAll 608 13200.3398 5.6130 24.9880 21.7111
MXNDArraySyncCopyToCPU 1296 10189.2041 0.1180 24.1980 7.8620
MXNDArrayGetDType 1296 0.4680 0.0000 0.0010 0.0004
MXNDArrayGetShape 3280 1.5160 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1376 12.0560 0.0070 0.0280 0.0088
MXNDArrayFree 15424 157.1690 0.0000 0.9140 0.0102
MXNet C API Calls 69456 556.8130 487.3580 556.8130 34.7275
MXNet C API Concurrency 138911 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1216 0.5720 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6080 3.1470 0.0000 0.0230 0.0005
MXNDArraySetGradState 6080 3.0690 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1216 0.5020 0.0000 0.0010 0.0004
MXAutogradBackwardEx 608 330.4140 0.5250 0.6810 0.5434
MXNDArrayGetContext 9552 4.1400 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20128 511.2550 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2432 5958.9790 0.7010 3.7500 2.4502
FullyConnected 4128 398.5580 0.0080 0.3430 0.0965
argmax 1376 27.1820 0.0050 0.0480 0.0198
_backward_pick 1216 19.1710 0.0130 0.0260 0.0158
_backward_mul_scalar 1216 5.2530 0.0030 0.0160 0.0043
DeleteVariable 51520 363.2060 0.0010 0.9010 0.0070
Flatten 1376 56.9510 0.0080 0.1120 0.0414
Activation 5504 256.1740 0.0030 1.9440 0.0465
SetValueOp 1216 7.8100 0.0050 0.0180 0.0064
pick 1216 9.5440 0.0060 0.0280 0.0078
Pooling 2752 3859.2930 0.0640 3.1720 1.4024
Convolution 2752 15616.5693 0.2260 13.2050 5.6746
CopyCPU2CPU 1376 7.8080 0.0030 0.0360 0.0057
log_softmax 1216 37.8240 0.0150 0.0550 0.0311
_backward_copy 1216 62.7030 0.0220 0.0760 0.0516
_backward_Convolution 2432 22504.5195 2.6370 12.5610 9.2535
_mul_scalar 1216 4.3410 0.0020 0.0230 0.0036
_backward_log_softmax 1216 20.5940 0.0090 0.0400 0.0169
mean 3808 73.0360 0.0080 0.0730 0.0192
_backward_mean 1216 13.3020 0.0080 0.1030 0.0109
_backward_FullyConnected 3648 693.1570 0.0280 0.5940 0.1900
WaitForVar 2592 23.0940 0.0060 0.0280 0.0089
broadcast_equal 1376 6.8570 0.0030 0.0110 0.0050
sgd_update 12160 119.7920 0.0020 0.0790 0.0099
_backward_Activation 4864 360.2140 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47466 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4134 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 608 2395.4890 0.3060 73.5850 3.9399
MXNDArrayCreateEx 689 23.5290 0.0310 0.0640 0.0341
MXNDArrayWaitAll 609 13225.2939 5.6130 24.9880 21.7164
MXNDArraySyncCopyToCPU 1298 10215.3096 0.1180 24.1980 7.8700
MXNDArrayGetDType 1298 0.4680 0.0000 0.0010 0.0004
MXNDArrayGetShape 3285 1.5180 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1378 12.0730 0.0070 0.0280 0.0088
MXNDArrayFree 15447 157.4050 0.0000 0.9140 0.0102
MXNet C API Calls 69563 556.9200 487.3580 556.9200 34.7810
MXNet C API Concurrency 139125 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1218 0.5720 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6090 3.1570 0.0000 0.0230 0.0005
MXNDArraySetGradState 6090 3.0730 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1218 0.5020 0.0000 0.0010 0.0004
MXAutogradBackwardEx 609 330.9540 0.5250 0.6810 0.5434
MXNDArrayGetContext 9566 4.1470 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20159 512.0430 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2436 5969.9570 0.7010 3.7500 2.4507
FullyConnected 4134 399.4450 0.0080 0.3430 0.0966
argmax 1378 27.2370 0.0050 0.0480 0.0198
_backward_pick 1218 19.2150 0.0130 0.0260 0.0158
_backward_mul_scalar 1218 5.2670 0.0030 0.0160 0.0043
DeleteVariable 51600 363.8120 0.0010 0.9010 0.0071
Flatten 1378 57.0510 0.0080 0.1120 0.0414
Activation 5512 256.5990 0.0030 1.9440 0.0466
SetValueOp 1218 7.8280 0.0050 0.0180 0.0064
pick 1218 9.5670 0.0060 0.0280 0.0079
Pooling 2756 3868.7849 0.0640 3.1720 1.4038
Convolution 2756 15654.9707 0.2260 13.2050 5.6803
CopyCPU2CPU 1378 7.8250 0.0030 0.0360 0.0057
log_softmax 1218 37.9040 0.0150 0.0550 0.0311
_backward_copy 1218 62.8290 0.0220 0.0760 0.0516
_backward_Convolution 2436 22547.7832 2.6370 12.5610 9.2561
_mul_scalar 1218 4.3530 0.0020 0.0230 0.0036
_backward_log_softmax 1218 20.6410 0.0090 0.0400 0.0169
mean 3814 73.1950 0.0080 0.0730 0.0192
_backward_mean 1218 13.3330 0.0080 0.1030 0.0109
_backward_FullyConnected 3654 694.8810 0.0280 0.5940 0.1902
WaitForVar 2596 23.1430 0.0060 0.0280 0.0089
broadcast_equal 1378 6.8710 0.0030 0.0110 0.0050
sgd_update 12180 120.0850 0.0020 0.0790 0.0099
_backward_Activation 4872 360.8020 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47540 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4140 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 609 2399.4150 0.3060 73.5850 3.9399
MXNDArrayCreateEx 690 23.5630 0.0310 0.0640 0.0341
MXNDArrayWaitAll 610 13247.1230 5.6130 24.9880 21.7166
MXNDArraySyncCopyToCPU 1300 10229.7275 0.1180 24.1980 7.8690
MXNDArrayGetDType 1300 0.4680 0.0000 0.0010 0.0004
MXNDArrayGetShape 3290 1.5190 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1380 12.0900 0.0070 0.0280 0.0088
MXNDArrayFree 15470 157.6430 0.0000 0.9140 0.0102
MXNet C API Calls 69670 557.0270 487.3580 557.0270 34.8345
MXNet C API Concurrency 139339 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1220 0.5730 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6100 3.1610 0.0000 0.0230 0.0005
MXNDArraySetGradState 6100 3.0770 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1220 0.5020 0.0000 0.0010 0.0004
MXAutogradBackwardEx 610 331.4890 0.5250 0.6810 0.5434
MXNDArrayGetContext 9580 4.1530 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20190 512.8470 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2440 5979.6558 0.7010 3.7500 2.4507
FullyConnected 4140 399.9910 0.0080 0.3430 0.0966
argmax 1380 27.2770 0.0050 0.0480 0.0198
_backward_pick 1220 19.2460 0.0130 0.0260 0.0158
_backward_mul_scalar 1220 5.2760 0.0030 0.0160 0.0043
DeleteVariable 51680 364.3680 0.0010 0.9010 0.0071
Flatten 1380 57.1310 0.0080 0.1120 0.0414
Activation 5520 256.9520 0.0030 1.9440 0.0465
SetValueOp 1220 7.8410 0.0050 0.0180 0.0064
pick 1220 9.5830 0.0060 0.0280 0.0079
Pooling 2760 3874.1780 0.0640 3.1720 1.4037
Convolution 2760 15677.1836 0.2260 13.2050 5.6801
CopyCPU2CPU 1380 7.8350 0.0030 0.0360 0.0057
log_softmax 1220 37.9640 0.0150 0.0550 0.0311
_backward_copy 1220 62.9340 0.0220 0.0760 0.0516
_backward_Convolution 2440 22584.8301 2.6370 12.5610 9.2561
_mul_scalar 1220 4.3600 0.0020 0.0230 0.0036
_backward_log_softmax 1220 20.6720 0.0090 0.0400 0.0169
mean 3820 73.3110 0.0080 0.0730 0.0192
_backward_mean 1220 13.3550 0.0080 0.1030 0.0109
_backward_FullyConnected 3660 695.9510 0.0280 0.5940 0.1902
WaitForVar 2600 23.1790 0.0060 0.0280 0.0089
broadcast_equal 1380 6.8810 0.0030 0.0110 0.0050
sgd_update 12200 120.2870 0.0020 0.0790 0.0099
_backward_Activation 4880 361.3520 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47614 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4146 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 610 2403.3259 0.3060 73.5850 3.9399
MXNDArrayCreateEx 691 23.5960 0.0310 0.0640 0.0341
MXNDArrayWaitAll 611 13268.8730 5.6130 24.9880 21.7167
MXNDArraySyncCopyToCPU 1302 10243.6680 0.1180 24.1980 7.8676
MXNDArrayGetDType 1302 0.4700 0.0000 0.0010 0.0004
MXNDArrayGetShape 3295 1.5210 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1382 12.1070 0.0070 0.0280 0.0088
MXNDArrayFree 15493 157.8890 0.0000 0.9140 0.0102
MXNet C API Calls 69777 557.1340 487.3580 557.1340 34.8880
MXNet C API Concurrency 139553 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1222 0.5740 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6110 3.1670 0.0000 0.0230 0.0005
MXNDArraySetGradState 6110 3.0820 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1222 0.5040 0.0000 0.0010 0.0004
MXAutogradBackwardEx 611 332.0580 0.5250 0.6810 0.5435
MXNDArrayGetContext 9594 4.1620 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20221 513.6350 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2444 5989.3628 0.7010 3.7500 2.4506
FullyConnected 4146 400.6190 0.0080 0.3430 0.0966
argmax 1382 27.3220 0.0050 0.0480 0.0198
_backward_pick 1222 19.2810 0.0130 0.0260 0.0158
_backward_mul_scalar 1222 5.2860 0.0030 0.0160 0.0043
DeleteVariable 51760 364.9300 0.0010 0.9010 0.0071
Flatten 1382 57.2130 0.0080 0.1120 0.0414
Activation 5528 257.3210 0.0030 1.9440 0.0465
SetValueOp 1222 7.8550 0.0050 0.0180 0.0064
pick 1222 9.6000 0.0060 0.0280 0.0079
Pooling 2764 3879.5320 0.0640 3.1720 1.4036
Convolution 2764 15699.4941 0.2260 13.2050 5.6800
CopyCPU2CPU 1382 7.8450 0.0030 0.0360 0.0057
log_softmax 1222 38.0270 0.0150 0.0550 0.0311
_backward_copy 1222 63.0370 0.0220 0.0760 0.0516
_backward_Convolution 2444 22621.6445 2.6370 12.5610 9.2560
_mul_scalar 1222 4.3670 0.0020 0.0230 0.0036
_backward_log_softmax 1222 20.7120 0.0090 0.0400 0.0169
mean 3826 73.4190 0.0080 0.0730 0.0192
_backward_mean 1222 13.3770 0.0080 0.1030 0.0109
_backward_FullyConnected 3666 697.1380 0.0280 0.5940 0.1902
WaitForVar 2604 23.2160 0.0060 0.0280 0.0089
broadcast_equal 1382 6.8920 0.0030 0.0110 0.0050
sgd_update 12220 120.5010 0.0020 0.0790 0.0099
_backward_Activation 4888 361.9220 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47688 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4152 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 611 2407.2500 0.3060 73.5850 3.9399
MXNDArrayCreateEx 692 23.6300 0.0310 0.0640 0.0341
MXNDArrayWaitAll 612 13290.5566 5.6130 24.9880 21.7166
MXNDArraySyncCopyToCPU 1304 10257.9609 0.1180 24.1980 7.8665
MXNDArrayGetDType 1304 0.4710 0.0000 0.0010 0.0004
MXNDArrayGetShape 3300 1.5230 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1384 12.1240 0.0070 0.0280 0.0088
MXNDArrayFree 15516 158.1180 0.0000 0.9140 0.0102
MXNet C API Calls 69884 557.2410 487.3580 557.2410 34.9415
MXNet C API Concurrency 139767 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1224 0.5750 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6120 3.1710 0.0000 0.0230 0.0005
MXNDArraySetGradState 6120 3.0880 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1224 0.5060 0.0000 0.0010 0.0004
MXAutogradBackwardEx 612 332.5970 0.5250 0.6810 0.5435
MXNDArrayGetContext 9608 4.1660 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20252 514.4420 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2448 5998.9990 0.7010 3.7500 2.4506
FullyConnected 4152 401.1480 0.0080 0.3430 0.0966
argmax 1384 27.3610 0.0050 0.0480 0.0198
_backward_pick 1224 19.3120 0.0130 0.0260 0.0158
_backward_mul_scalar 1224 5.2930 0.0030 0.0160 0.0043
DeleteVariable 51840 365.4740 0.0010 0.9010 0.0071
Flatten 1384 57.2960 0.0080 0.1120 0.0414
Activation 5536 257.6710 0.0030 1.9440 0.0465
SetValueOp 1224 7.8680 0.0050 0.0180 0.0064
pick 1224 9.6150 0.0060 0.0280 0.0079
Pooling 2768 3884.9180 0.0640 3.1720 1.4035
Convolution 2768 15721.5605 0.2260 13.2050 5.6798
CopyCPU2CPU 1384 7.8540 0.0030 0.0360 0.0057
log_softmax 1224 38.0870 0.0150 0.0550 0.0311
_backward_copy 1224 63.1390 0.0220 0.0760 0.0516
_backward_Convolution 2448 22658.4453 2.6370 12.5610 9.2559
_mul_scalar 1224 4.3740 0.0020 0.0230 0.0036
_backward_log_softmax 1224 20.7430 0.0090 0.0400 0.0169
mean 3832 73.5340 0.0080 0.0730 0.0192
_backward_mean 1224 13.3980 0.0080 0.1030 0.0109
_backward_FullyConnected 3672 698.2110 0.0280 0.5940 0.1901
WaitForVar 2608 23.2510 0.0060 0.0280 0.0089
broadcast_equal 1384 6.9020 0.0030 0.0110 0.0050
sgd_update 12240 120.6960 0.0020 0.0790 0.0099
_backward_Activation 4896 362.4500 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47762 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4158 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 612 2411.1580 0.3060 73.5850 3.9398
MXNDArrayCreateEx 693 23.6640 0.0310 0.0640 0.0341
MXNDArrayWaitAll 613 13315.5781 5.6130 25.0210 21.7220
MXNDArraySyncCopyToCPU 1306 10284.1035 0.1180 24.1980 7.8745
MXNDArrayGetDType 1306 0.4720 0.0000 0.0010 0.0004
MXNDArrayGetShape 3305 1.5240 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1386 12.1410 0.0070 0.0280 0.0088
MXNDArrayFree 15539 158.3620 0.0000 0.9140 0.0102
MXNet C API Calls 69991 557.3480 487.3580 557.3480 34.9950
MXNet C API Concurrency 139981 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1226 0.5760 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6130 3.1770 0.0000 0.0230 0.0005
MXNDArraySetGradState 6130 3.0930 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1226 0.5080 0.0000 0.0010 0.0004
MXAutogradBackwardEx 613 333.1350 0.5250 0.6810 0.5435
MXNDArrayGetContext 9622 4.1720 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20283 515.2260 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2452 6009.9111 0.7010 3.7500 2.4510
FullyConnected 4158 402.0360 0.0080 0.3430 0.0967
argmax 1386 27.4140 0.0050 0.0480 0.0198
_backward_pick 1226 19.3550 0.0130 0.0260 0.0158
_backward_mul_scalar 1226 5.3080 0.0030 0.0160 0.0043
DeleteVariable 51920 366.0790 0.0010 0.9010 0.0071
Flatten 1386 57.3940 0.0080 0.1120 0.0414
Activation 5544 258.0920 0.0030 1.9440 0.0466
SetValueOp 1226 7.8870 0.0050 0.0180 0.0064
pick 1226 9.6380 0.0060 0.0280 0.0079
Pooling 2772 3894.4460 0.0640 3.1720 1.4049
Convolution 2772 15759.9043 0.2260 13.2050 5.6854
CopyCPU2CPU 1386 7.8670 0.0030 0.0360 0.0057
log_softmax 1226 38.1660 0.0150 0.0550 0.0311
_backward_copy 1226 63.2610 0.0220 0.0760 0.0516
_backward_Convolution 2452 22701.8516 2.6370 12.5610 9.2585
_mul_scalar 1226 4.3870 0.0020 0.0230 0.0036
_backward_log_softmax 1226 20.7900 0.0090 0.0400 0.0170
mean 3838 73.6900 0.0080 0.0730 0.0192
_backward_mean 1226 13.4300 0.0080 0.1030 0.0110
_backward_FullyConnected 3678 699.9300 0.0280 0.5940 0.1903
WaitForVar 2612 23.3020 0.0060 0.0280 0.0089
broadcast_equal 1386 6.9180 0.0030 0.0110 0.0050
sgd_update 12260 120.9520 0.0020 0.0790 0.0099
_backward_Activation 4904 363.1080 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47836 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4164 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 613 2414.8269 0.3060 73.5850 3.9394
MXNDArrayCreateEx 694 23.6990 0.0310 0.0640 0.0341
MXNDArrayWaitAll 614 13337.4521 5.6130 25.0210 21.7222
MXNDArraySyncCopyToCPU 1308 10298.5137 0.1180 24.1980 7.8735
MXNDArrayGetDType 1308 0.4730 0.0000 0.0010 0.0004
MXNDArrayGetShape 3310 1.5250 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1388 12.1580 0.0070 0.0280 0.0088
MXNDArrayFree 15562 158.6030 0.0000 0.9140 0.0102
MXNet C API Calls 70098 557.4550 487.3580 557.4550 35.0485
MXNet C API Concurrency 140195 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1228 0.5760 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6140 3.1830 0.0000 0.0230 0.0005
MXNDArraySetGradState 6140 3.0960 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1228 0.5080 0.0000 0.0010 0.0004
MXAutogradBackwardEx 614 333.6790 0.5250 0.6810 0.5435
MXNDArrayGetContext 9636 4.1770 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20314 516.0140 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2456 6019.5601 0.7010 3.7500 2.4510
FullyConnected 4164 402.5610 0.0080 0.3430 0.0967
argmax 1388 27.4520 0.0050 0.0480 0.0198
_backward_pick 1228 19.3870 0.0130 0.0260 0.0158
_backward_mul_scalar 1228 5.3160 0.0030 0.0160 0.0043
DeleteVariable 52000 366.6220 0.0010 0.9010 0.0071
Flatten 1388 57.4750 0.0080 0.1120 0.0414
Activation 5552 258.4470 0.0030 1.9440 0.0466
SetValueOp 1228 7.9010 0.0050 0.0180 0.0064
pick 1228 9.6530 0.0060 0.0280 0.0079
Pooling 2776 3899.8230 0.0640 3.1720 1.4048
Convolution 2776 15782.1885 0.2260 13.2050 5.6852
CopyCPU2CPU 1388 7.8770 0.0030 0.0360 0.0057
log_softmax 1228 38.2260 0.0150 0.0550 0.0311
_backward_copy 1228 63.3650 0.0220 0.0760 0.0516
_backward_Convolution 2456 22738.9707 2.6370 12.5610 9.2585
_mul_scalar 1228 4.3940 0.0020 0.0230 0.0036
_backward_log_softmax 1228 20.8220 0.0090 0.0400 0.0170
mean 3844 73.8050 0.0080 0.0730 0.0192
_backward_mean 1228 13.4520 0.0080 0.1030 0.0110
_backward_FullyConnected 3684 701.0150 0.0280 0.5940 0.1903
WaitForVar 2616 23.3370 0.0060 0.0280 0.0089
broadcast_equal 1388 6.9280 0.0030 0.0110 0.0050
sgd_update 12280 121.1480 0.0020 0.0790 0.0099
_backward_Activation 4912 363.6580 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47910 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4170 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 614 2418.7351 0.3060 73.5850 3.9393
MXNDArrayCreateEx 695 23.7320 0.0310 0.0640 0.0341
MXNDArrayWaitAll 615 13359.1836 5.6130 25.0210 21.7222
MXNDArraySyncCopyToCPU 1310 10313.1221 0.1180 24.1980 7.8726
MXNDArrayGetDType 1310 0.4740 0.0000 0.0010 0.0004
MXNDArrayGetShape 3315 1.5270 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1390 12.1770 0.0070 0.0280 0.0088
MXNDArrayFree 15585 158.8490 0.0000 0.9140 0.0102
MXNet C API Calls 70205 557.5620 487.3580 557.5620 35.1020
MXNet C API Concurrency 140409 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1230 0.5760 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6150 3.1900 0.0000 0.0230 0.0005
MXNDArraySetGradState 6150 3.1010 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1230 0.5080 0.0000 0.0010 0.0004
MXAutogradBackwardEx 615 334.2240 0.5250 0.6810 0.5435
MXNDArrayGetContext 9650 4.1790 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20345 516.8180 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2460 6029.3721 0.7010 3.7500 2.4510
FullyConnected 4170 403.1620 0.0080 0.3430 0.0967
argmax 1390 27.4900 0.0050 0.0480 0.0198
_backward_pick 1230 19.4350 0.0130 0.0320 0.0158
_backward_mul_scalar 1230 5.3260 0.0030 0.0160 0.0043
DeleteVariable 52080 367.1880 0.0010 0.9010 0.0071
Flatten 1390 57.5550 0.0080 0.1120 0.0414
Activation 5560 258.8470 0.0030 1.9440 0.0466
SetValueOp 1230 7.9160 0.0050 0.0180 0.0064
pick 1230 9.6700 0.0060 0.0280 0.0079
Pooling 2780 3905.2390 0.0640 3.1720 1.4048
Convolution 2780 15804.4258 0.2260 13.2050 5.6850
CopyCPU2CPU 1390 7.8870 0.0030 0.0360 0.0057
log_softmax 1230 38.2850 0.0150 0.0550 0.0311
_backward_copy 1230 63.4680 0.0220 0.0760 0.0516
_backward_Convolution 2460 22775.7422 2.6370 12.5610 9.2584
_mul_scalar 1230 4.4010 0.0020 0.0230 0.0036
_backward_log_softmax 1230 20.8640 0.0090 0.0400 0.0170
mean 3850 73.9110 0.0080 0.0730 0.0192
_backward_mean 1230 13.4740 0.0080 0.1030 0.0110
_backward_FullyConnected 3690 702.1980 0.0280 0.5940 0.1903
WaitForVar 2620 23.3740 0.0060 0.0280 0.0089
broadcast_equal 1390 6.9390 0.0030 0.0110 0.0050
sgd_update 12300 121.3170 0.0020 0.0790 0.0099
_backward_Activation 4920 364.2900 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 47984 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4176 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 615 2422.6521 0.3060 73.5850 3.9393
MXNDArrayCreateEx 696 23.7670 0.0310 0.0640 0.0341
MXNDArrayWaitAll 616 13380.9111 5.6130 25.0210 21.7223
MXNDArraySyncCopyToCPU 1312 10326.9980 0.1180 24.1980 7.8712
MXNDArrayGetDType 1312 0.4750 0.0000 0.0010 0.0004
MXNDArrayGetShape 3320 1.5310 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1392 12.1950 0.0070 0.0280 0.0088
MXNDArrayFree 15608 159.0810 0.0000 0.9140 0.0102
MXNet C API Calls 70312 557.6690 487.3580 557.6690 35.1555
MXNet C API Concurrency 140623 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1232 0.5770 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6160 3.1920 0.0000 0.0230 0.0005
MXNDArraySetGradState 6160 3.1040 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1232 0.5100 0.0000 0.0010 0.0004
MXAutogradBackwardEx 616 334.7640 0.5250 0.6810 0.5434
MXNDArrayGetContext 9664 4.1860 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20376 517.6110 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2464 6039.1279 0.7010 3.7500 2.4509
FullyConnected 4176 403.6890 0.0080 0.3430 0.0967
argmax 1392 27.5280 0.0050 0.0480 0.0198
_backward_pick 1232 19.4690 0.0130 0.0320 0.0158
_backward_mul_scalar 1232 5.3350 0.0030 0.0160 0.0043
DeleteVariable 52160 367.7320 0.0010 0.9010 0.0071
Flatten 1392 57.6380 0.0080 0.1120 0.0414
Activation 5568 259.2010 0.0030 1.9440 0.0466
SetValueOp 1232 7.9290 0.0050 0.0180 0.0064
pick 1232 9.6840 0.0060 0.0280 0.0079
Pooling 2784 3910.5791 0.0640 3.1720 1.4047
Convolution 2784 15826.5713 0.2260 13.2050 5.6848
CopyCPU2CPU 1392 7.8960 0.0030 0.0360 0.0057
log_softmax 1232 38.3450 0.0150 0.0550 0.0311
_backward_copy 1232 63.5720 0.0220 0.0760 0.0516
_backward_Convolution 2464 22812.5020 2.6370 12.5610 9.2583
_mul_scalar 1232 4.4080 0.0020 0.0230 0.0036
_backward_log_softmax 1232 20.8960 0.0090 0.0400 0.0170
mean 3856 74.0490 0.0080 0.0730 0.0192
_backward_mean 1232 13.4970 0.0080 0.1030 0.0110
_backward_FullyConnected 3696 703.2790 0.0280 0.5940 0.1903
WaitForVar 2624 23.4080 0.0060 0.0280 0.0089
broadcast_equal 1392 6.9490 0.0030 0.0110 0.0050
sgd_update 12320 121.5180 0.0020 0.0790 0.0099
_backward_Activation 4928 364.8540 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48058 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4182 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 616 2426.5630 0.3060 73.5850 3.9392
MXNDArrayCreateEx 697 23.8010 0.0310 0.0640 0.0341
MXNDArrayWaitAll 617 13405.7129 5.6130 25.0210 21.7272
MXNDArraySyncCopyToCPU 1314 10352.9766 0.1180 24.1980 7.8790
MXNDArrayGetDType 1314 0.4750 0.0000 0.0010 0.0004
MXNDArrayGetShape 3325 1.5350 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1394 12.2130 0.0070 0.0280 0.0088
MXNDArrayFree 15631 159.3310 0.0000 0.9140 0.0102
MXNet C API Calls 70419 557.7760 487.3580 557.7760 35.2090
MXNet C API Concurrency 140837 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1234 0.5790 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6170 3.1970 0.0000 0.0230 0.0005
MXNDArraySetGradState 6170 3.1080 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1234 0.5100 0.0000 0.0010 0.0004
MXAutogradBackwardEx 617 335.3290 0.5250 0.6810 0.5435
MXNDArrayGetContext 9678 4.1890 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20407 518.4140 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2468 6050.0381 0.7010 3.7500 2.4514
FullyConnected 4182 404.5840 0.0080 0.3430 0.0967
argmax 1394 27.5830 0.0050 0.0480 0.0198
_backward_pick 1234 19.5120 0.0130 0.0320 0.0158
_backward_mul_scalar 1234 5.3500 0.0030 0.0160 0.0043
DeleteVariable 52240 368.3670 0.0010 0.9010 0.0071
Flatten 1394 57.7590 0.0080 0.1120 0.0414
Activation 5576 259.6240 0.0030 1.9440 0.0466
SetValueOp 1234 7.9460 0.0050 0.0180 0.0064
pick 1234 9.7070 0.0060 0.0280 0.0079
Pooling 2788 3919.9951 0.0640 3.1720 1.4060
Convolution 2788 15864.9971 0.2260 13.2050 5.6905
CopyCPU2CPU 1394 7.9090 0.0030 0.0360 0.0057
log_softmax 1234 38.4240 0.0150 0.0550 0.0311
_backward_copy 1234 63.6900 0.0220 0.0760 0.0516
_backward_Convolution 2468 22855.4668 2.6370 12.5610 9.2607
_mul_scalar 1234 4.4190 0.0020 0.0230 0.0036
_backward_log_softmax 1234 20.9450 0.0090 0.0400 0.0170
mean 3862 74.2100 0.0080 0.0730 0.0192
_backward_mean 1234 13.5280 0.0080 0.1030 0.0110
_backward_FullyConnected 3702 705.0340 0.0280 0.5940 0.1904
WaitForVar 2628 23.4540 0.0060 0.0280 0.0089
broadcast_equal 1394 6.9640 0.0030 0.0110 0.0050
sgd_update 12340 121.7610 0.0020 0.0790 0.0099
_backward_Activation 4936 365.5210 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48132 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4188 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 617 2430.4719 0.3060 73.5850 3.9392
MXNDArrayCreateEx 698 23.8350 0.0310 0.0640 0.0341
MXNDArrayWaitAll 618 13427.5889 5.6130 25.0210 21.7275
MXNDArraySyncCopyToCPU 1316 10367.3779 0.1180 24.1980 7.8779
MXNDArrayGetDType 1316 0.4750 0.0000 0.0010 0.0004
MXNDArrayGetShape 3330 1.5370 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1396 12.2310 0.0070 0.0280 0.0088
MXNDArrayFree 15654 159.5660 0.0000 0.9140 0.0102
MXNet C API Calls 70526 557.8830 487.3580 557.8830 35.2625
MXNet C API Concurrency 141051 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1236 0.5800 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6180 3.2020 0.0000 0.0230 0.0005
MXNDArraySetGradState 6180 3.1120 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1236 0.5110 0.0000 0.0010 0.0004
MXAutogradBackwardEx 618 335.8700 0.5250 0.6810 0.5435
MXNDArrayGetContext 9692 4.1930 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20438 519.2010 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2472 6059.8179 0.7010 3.7500 2.4514
FullyConnected 4188 405.1050 0.0080 0.3430 0.0967
argmax 1396 27.6230 0.0050 0.0480 0.0198
_backward_pick 1236 19.5420 0.0130 0.0320 0.0158
_backward_mul_scalar 1236 5.3590 0.0030 0.0160 0.0043
DeleteVariable 52320 368.9070 0.0010 0.9010 0.0071
Flatten 1396 57.8410 0.0080 0.1120 0.0414
Activation 5584 260.0070 0.0030 1.9440 0.0466
SetValueOp 1236 7.9590 0.0050 0.0180 0.0064
pick 1236 9.7230 0.0060 0.0280 0.0079
Pooling 2792 3925.3630 0.0640 3.1720 1.4059
Convolution 2792 15887.2451 0.2260 13.2050 5.6903
CopyCPU2CPU 1396 7.9190 0.0030 0.0360 0.0057
log_softmax 1236 38.4840 0.0150 0.0550 0.0311
_backward_copy 1236 63.7920 0.0220 0.0760 0.0516
_backward_Convolution 2472 22892.5293 2.6370 12.5610 9.2607
_mul_scalar 1236 4.4260 0.0020 0.0230 0.0036
_backward_log_softmax 1236 20.9780 0.0090 0.0400 0.0170
mean 3868 74.3230 0.0080 0.0730 0.0192
_backward_mean 1236 13.5500 0.0080 0.1030 0.0110
_backward_FullyConnected 3708 706.1050 0.0280 0.5940 0.1904
WaitForVar 2632 23.4890 0.0060 0.0280 0.0089
broadcast_equal 1396 6.9740 0.0030 0.0110 0.0050
sgd_update 12360 121.9100 0.0020 0.0790 0.0099
_backward_Activation 4944 366.1470 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48206 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4194 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 618 2434.3889 0.3060 73.5850 3.9391
MXNDArrayCreateEx 699 23.8690 0.0310 0.0640 0.0341
MXNDArrayWaitAll 619 13449.3252 5.6130 25.0210 21.7275
MXNDArraySyncCopyToCPU 1318 10381.9014 0.1180 24.1980 7.8770
MXNDArrayGetDType 1318 0.4750 0.0000 0.0010 0.0004
MXNDArrayGetShape 3335 1.5390 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1398 12.2490 0.0070 0.0280 0.0088
MXNDArrayFree 15677 159.8000 0.0000 0.9140 0.0102
MXNet C API Calls 70633 557.9900 487.3580 557.9900 35.3160
MXNet C API Concurrency 141265 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1238 0.5810 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6190 3.2050 0.0000 0.0230 0.0005
MXNDArraySetGradState 6190 3.1170 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1238 0.5130 0.0000 0.0010 0.0004
MXAutogradBackwardEx 619 336.4060 0.5250 0.6810 0.5435
MXNDArrayGetContext 9706 4.1980 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20469 519.9810 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2476 6069.5151 0.7010 3.7500 2.4513
FullyConnected 4194 405.7160 0.0080 0.3430 0.0967
argmax 1398 27.6650 0.0050 0.0480 0.0198
_backward_pick 1238 19.5730 0.0130 0.0320 0.0158
_backward_mul_scalar 1238 5.3680 0.0030 0.0160 0.0043
DeleteVariable 52400 369.4790 0.0010 0.9010 0.0071
Flatten 1398 57.9220 0.0080 0.1120 0.0414
Activation 5592 260.3820 0.0030 1.9440 0.0466
SetValueOp 1238 7.9730 0.0050 0.0180 0.0064
pick 1238 9.7420 0.0060 0.0280 0.0079
Pooling 2796 3930.6919 0.0640 3.1720 1.4058
Convolution 2796 15909.5146 0.2260 13.2050 5.6901
CopyCPU2CPU 1398 7.9290 0.0030 0.0360 0.0057
log_softmax 1238 38.5430 0.0150 0.0550 0.0311
_backward_copy 1238 63.8920 0.0220 0.0760 0.0516
_backward_Convolution 2476 22929.3594 2.6370 12.5610 9.2606
_mul_scalar 1238 4.4330 0.0020 0.0230 0.0036
_backward_log_softmax 1238 21.0200 0.0090 0.0400 0.0170
mean 3874 74.4280 0.0080 0.0730 0.0192
_backward_mean 1238 13.5730 0.0080 0.1030 0.0110
_backward_FullyConnected 3714 707.2910 0.0280 0.5940 0.1904
WaitForVar 2636 23.5250 0.0060 0.0280 0.0089
broadcast_equal 1398 6.9840 0.0030 0.0110 0.0050
sgd_update 12380 122.1420 0.0020 0.0790 0.0099
_backward_Activation 4952 366.7150 0.0070 0.3900 0.0741
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48280 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4200 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 619 2438.5200 0.3060 73.5850 3.9395
MXNDArrayCreateEx 700 23.9030 0.0310 0.0640 0.0341
MXNDArrayWaitAll 620 13471.0879 5.6130 25.0210 21.7276
MXNDArraySyncCopyToCPU 1320 10396.3291 0.1180 24.1980 7.8760
MXNDArrayGetDType 1320 0.4750 0.0000 0.0010 0.0004
MXNDArrayGetShape 3340 1.5410 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1400 12.2660 0.0070 0.0280 0.0088
MXNDArrayFree 15700 160.0300 0.0000 0.9140 0.0102
MXNet C API Calls 70740 558.0970 487.3580 558.0970 35.3695
MXNet C API Concurrency 141479 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1240 0.5830 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6200 3.2090 0.0000 0.0230 0.0005
MXNDArraySetGradState 6200 3.1230 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1240 0.5140 0.0000 0.0010 0.0004
MXAutogradBackwardEx 620 336.9680 0.5250 0.6810 0.5435
MXNDArrayGetContext 9720 4.2030 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20500 520.7590 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2480 6079.2139 0.7010 3.7500 2.4513
FullyConnected 4200 406.3350 0.0080 0.3430 0.0967
argmax 1400 27.7080 0.0050 0.0480 0.0198
_backward_pick 1240 19.6040 0.0130 0.0320 0.0158
_backward_mul_scalar 1240 5.3760 0.0030 0.0160 0.0043
DeleteVariable 52480 370.0240 0.0010 0.9010 0.0071
Flatten 1400 58.0020 0.0080 0.1120 0.0414
Activation 5600 260.7540 0.0030 1.9440 0.0466
SetValueOp 1240 7.9850 0.0050 0.0180 0.0064
pick 1240 9.7570 0.0060 0.0280 0.0079
Pooling 2800 3936.1130 0.0640 3.1720 1.4058
Convolution 2800 15931.6133 0.2260 13.2050 5.6899
CopyCPU2CPU 1400 7.9400 0.0030 0.0360 0.0057
log_softmax 1240 38.6030 0.0150 0.0550 0.0311
_backward_copy 1240 63.9940 0.0220 0.0760 0.0516
_backward_Convolution 2480 22966.1602 2.6370 12.5610 9.2605
_mul_scalar 1240 4.4400 0.0020 0.0230 0.0036
_backward_log_softmax 1240 21.0620 0.0090 0.0400 0.0170
mean 3880 74.5320 0.0080 0.0730 0.0192
_backward_mean 1240 13.5950 0.0080 0.1030 0.0110
_backward_FullyConnected 3720 708.4650 0.0280 0.5940 0.1904
WaitForVar 2640 23.5610 0.0060 0.0280 0.0089
broadcast_equal 1400 6.9950 0.0030 0.0110 0.0050
sgd_update 12400 122.3720 0.0020 0.0790 0.0099
_backward_Activation 4960 367.2730 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48354 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4206 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 620 2442.4241 0.3060 73.5850 3.9394
MXNDArrayCreateEx 701 23.9370 0.0310 0.0640 0.0341
MXNDArrayWaitAll 621 13496.1494 5.6130 25.0610 21.7329
MXNDArraySyncCopyToCPU 1322 10421.9121 0.1180 24.1980 7.8834
MXNDArrayGetDType 1322 0.4750 0.0000 0.0010 0.0004
MXNDArrayGetShape 3345 1.5420 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1402 12.2840 0.0070 0.0280 0.0088
MXNDArrayFree 15723 160.2730 0.0000 0.9140 0.0102
MXNet C API Calls 70847 558.2040 487.3580 558.2040 35.4230
MXNet C API Concurrency 141693 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1242 0.5830 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6210 3.2140 0.0000 0.0230 0.0005
MXNDArraySetGradState 6210 3.1280 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1242 0.5150 0.0000 0.0010 0.0004
MXAutogradBackwardEx 621 337.5100 0.5250 0.6810 0.5435
MXNDArrayGetContext 9734 4.2100 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20531 521.5490 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2484 6090.2969 0.7010 3.7500 2.4518
FullyConnected 4206 407.1970 0.0080 0.3430 0.0968
argmax 1402 27.7640 0.0050 0.0480 0.0198
_backward_pick 1242 19.6470 0.0130 0.0320 0.0158
_backward_mul_scalar 1242 5.3910 0.0030 0.0160 0.0043
DeleteVariable 52560 370.6610 0.0010 0.9010 0.0071
Flatten 1402 58.1030 0.0080 0.1120 0.0414
Activation 5608 261.1750 0.0030 1.9440 0.0466
SetValueOp 1242 8.0040 0.0050 0.0180 0.0064
pick 1242 9.7790 0.0060 0.0280 0.0079
Pooling 2804 3945.6060 0.0640 3.1720 1.4071
Convolution 2804 15969.9365 0.2260 13.2050 5.6954
CopyCPU2CPU 1402 7.9520 0.0030 0.0360 0.0057
log_softmax 1242 38.6830 0.0150 0.0550 0.0311
_backward_copy 1242 64.1130 0.0220 0.0760 0.0516
_backward_Convolution 2484 23009.5840 2.6370 12.5610 9.2631
_mul_scalar 1242 4.4520 0.0020 0.0230 0.0036
_backward_log_softmax 1242 21.1090 0.0090 0.0400 0.0170
mean 3886 74.6900 0.0080 0.0730 0.0192
_backward_mean 1242 13.6260 0.0080 0.1030 0.0110
_backward_FullyConnected 3726 710.1840 0.0280 0.5940 0.1906
WaitForVar 2644 23.6110 0.0060 0.0280 0.0089
broadcast_equal 1402 7.0090 0.0030 0.0110 0.0050
sgd_update 12420 122.6610 0.0020 0.0790 0.0099
_backward_Activation 4968 367.8630 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48428 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4212 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 621 2446.4021 0.3060 73.5850 3.9395
MXNDArrayCreateEx 702 23.9720 0.0310 0.0640 0.0341
MXNDArrayWaitAll 622 13518.0410 5.6130 25.0610 21.7332
MXNDArraySyncCopyToCPU 1324 10436.3203 0.1180 24.1980 7.8824
MXNDArrayGetDType 1324 0.4760 0.0000 0.0010 0.0004
MXNDArrayGetShape 3350 1.5630 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1404 12.3020 0.0070 0.0280 0.0088
MXNDArrayFree 15746 160.5090 0.0000 0.9140 0.0102
MXNet C API Calls 70954 558.3110 487.3580 558.3110 35.4765
MXNet C API Concurrency 141907 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1244 0.5850 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6220 3.2200 0.0000 0.0230 0.0005
MXNDArraySetGradState 6220 3.1320 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1244 0.5160 0.0000 0.0010 0.0004
MXAutogradBackwardEx 622 338.0500 0.5250 0.6810 0.5435
MXNDArrayGetContext 9748 4.2170 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20562 522.3360 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2488 6100.0171 0.7010 3.7500 2.4518
FullyConnected 4212 407.7440 0.0080 0.3430 0.0968
argmax 1404 27.8020 0.0050 0.0480 0.0198
_backward_pick 1244 19.6770 0.0130 0.0320 0.0158
_backward_mul_scalar 1244 5.4010 0.0030 0.0160 0.0043
DeleteVariable 52640 371.1990 0.0010 0.9010 0.0071
Flatten 1404 58.1830 0.0080 0.1120 0.0414
Activation 5616 261.5280 0.0030 1.9440 0.0466
SetValueOp 1244 8.0160 0.0050 0.0180 0.0064
pick 1244 9.7950 0.0060 0.0280 0.0079
Pooling 2808 3950.9360 0.0640 3.1720 1.4070
Convolution 2808 15992.2002 0.2260 13.2050 5.6952
CopyCPU2CPU 1404 7.9620 0.0030 0.0360 0.0057
log_softmax 1244 38.7430 0.0150 0.0550 0.0311
_backward_copy 1244 64.2160 0.0220 0.0760 0.0516
_backward_Convolution 2488 23046.7285 2.6370 12.5610 9.2632
_mul_scalar 1244 4.4590 0.0020 0.0230 0.0036
_backward_log_softmax 1244 21.1410 0.0090 0.0400 0.0170
mean 3892 74.8080 0.0080 0.0730 0.0192
_backward_mean 1244 13.6480 0.0080 0.1030 0.0110
_backward_FullyConnected 3732 711.2840 0.0280 0.5940 0.1906
WaitForVar 2648 23.6460 0.0060 0.0280 0.0089
broadcast_equal 1404 7.0190 0.0030 0.0110 0.0050
sgd_update 12440 122.8580 0.0020 0.0790 0.0099
_backward_Activation 4976 368.4110 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48502 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4218 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 622 2450.3210 0.3060 73.5850 3.9394
MXNDArrayCreateEx 703 24.0060 0.0310 0.0640 0.0341
MXNDArrayWaitAll 623 13539.7842 5.6130 25.0610 21.7332
MXNDArraySyncCopyToCPU 1326 10450.8965 0.1180 24.1980 7.8815
MXNDArrayGetDType 1326 0.4770 0.0000 0.0010 0.0004
MXNDArrayGetShape 3355 1.5670 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1406 12.3190 0.0070 0.0280 0.0088
MXNDArrayFree 15769 160.7460 0.0000 0.9140 0.0102
MXNet C API Calls 71061 558.4180 487.3580 558.4180 35.5300
MXNet C API Concurrency 142121 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1246 0.5870 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6230 3.2240 0.0000 0.0230 0.0005
MXNDArraySetGradState 6230 3.1400 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1246 0.5170 0.0000 0.0010 0.0004
MXAutogradBackwardEx 623 338.5930 0.5250 0.6810 0.5435
MXNDArrayGetContext 9762 4.2230 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20593 523.1110 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2492 6109.7329 0.7010 3.7500 2.4517
FullyConnected 4218 408.3580 0.0080 0.3430 0.0968
argmax 1406 27.8460 0.0050 0.0480 0.0198
_backward_pick 1246 19.7090 0.0130 0.0320 0.0158
_backward_mul_scalar 1246 5.4100 0.0030 0.0160 0.0043
DeleteVariable 52720 371.7700 0.0010 0.9010 0.0071
Flatten 1406 58.2620 0.0080 0.1120 0.0414
Activation 5624 261.8970 0.0030 1.9440 0.0466
SetValueOp 1246 8.0290 0.0050 0.0180 0.0064
pick 1246 9.8100 0.0060 0.0280 0.0079
Pooling 2812 3956.3730 0.0640 3.1720 1.4070
Convolution 2812 16014.4746 0.2260 13.2050 5.6950
CopyCPU2CPU 1406 7.9720 0.0030 0.0360 0.0057
log_softmax 1246 38.8030 0.0150 0.0550 0.0311
_backward_copy 1246 64.3190 0.0220 0.0760 0.0516
_backward_Convolution 2492 23083.5840 2.6370 12.5610 9.2631
_mul_scalar 1246 4.4660 0.0020 0.0230 0.0036
_backward_log_softmax 1246 21.1830 0.0090 0.0400 0.0170
mean 3898 74.9120 0.0080 0.0730 0.0192
_backward_mean 1246 13.6700 0.0080 0.1030 0.0110
_backward_FullyConnected 3738 712.4590 0.0280 0.5940 0.1906
WaitForVar 2652 23.6830 0.0060 0.0280 0.0089
broadcast_equal 1406 7.0290 0.0030 0.0110 0.0050
sgd_update 12460 123.0740 0.0020 0.0790 0.0099
_backward_Activation 4984 368.9740 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48576 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4224 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 623 2454.2319 0.3060 73.5850 3.9394
MXNDArrayCreateEx 704 24.0440 0.0310 0.0640 0.0342
MXNDArrayWaitAll 624 13561.4912 5.6130 25.0610 21.7332
MXNDArraySyncCopyToCPU 1328 10465.2930 0.1180 24.1980 7.8805
MXNDArrayGetDType 1328 0.4770 0.0000 0.0010 0.0004
MXNDArrayGetShape 3360 1.5700 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1408 12.3380 0.0070 0.0280 0.0088
MXNDArrayFree 15792 160.9860 0.0000 0.9140 0.0102
MXNet C API Calls 71168 558.5250 487.3580 558.5250 35.5835
MXNet C API Concurrency 142335 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1248 0.5870 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6240 3.2290 0.0000 0.0230 0.0005
MXNDArraySetGradState 6240 3.1430 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1248 0.5170 0.0000 0.0010 0.0004
MXAutogradBackwardEx 624 339.1440 0.5250 0.6810 0.5435
MXNDArrayGetContext 9776 4.2280 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20624 523.9150 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2496 6119.3008 0.7010 3.7500 2.4516
FullyConnected 4224 408.9880 0.0080 0.3430 0.0968
argmax 1408 27.8890 0.0050 0.0480 0.0198
_backward_pick 1248 19.7390 0.0130 0.0320 0.0158
_backward_mul_scalar 1248 5.4190 0.0030 0.0160 0.0043
DeleteVariable 52800 372.3700 0.0010 0.9010 0.0071
Flatten 1408 58.3430 0.0080 0.1120 0.0414
Activation 5632 262.2720 0.0030 1.9440 0.0466
SetValueOp 1248 8.0420 0.0050 0.0180 0.0064
pick 1248 9.8260 0.0060 0.0280 0.0079
Pooling 2816 3961.7329 0.0640 3.1720 1.4069
Convolution 2816 16036.5869 0.2260 13.2050 5.6948
CopyCPU2CPU 1408 7.9830 0.0030 0.0360 0.0057
log_softmax 1248 38.8630 0.0150 0.0550 0.0311
_backward_copy 1248 64.4200 0.0220 0.0760 0.0516
_backward_Convolution 2496 23120.3828 2.6370 12.5610 9.2630
_mul_scalar 1248 4.4730 0.0020 0.0230 0.0036
_backward_log_softmax 1248 21.2250 0.0090 0.0400 0.0170
mean 3904 75.0420 0.0080 0.0730 0.0192
_backward_mean 1248 13.6910 0.0080 0.1030 0.0110
_backward_FullyConnected 3744 713.6470 0.0280 0.5940 0.1906
WaitForVar 2656 23.7200 0.0060 0.0280 0.0089
broadcast_equal 1408 7.0380 0.0030 0.0110 0.0050
sgd_update 12480 123.2920 0.0020 0.0790 0.0099
_backward_Activation 4992 369.5540 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48650 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4230 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 624 2458.1709 0.3060 73.5850 3.9394
MXNDArrayCreateEx 705 24.0780 0.0310 0.0640 0.0342
MXNDArrayWaitAll 625 13583.3359 5.6130 25.0610 21.7333
MXNDArraySyncCopyToCPU 1330 10479.6396 0.1180 24.1980 7.8794
MXNDArrayGetDType 1330 0.4790 0.0000 0.0010 0.0004
MXNDArrayGetShape 3365 1.5710 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1410 12.3570 0.0070 0.0280 0.0088
MXNDArrayFree 15815 161.2210 0.0000 0.9140 0.0102
MXNet C API Calls 71275 558.6320 487.3580 558.6320 35.6370
MXNet C API Concurrency 142549 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1250 0.5880 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6250 3.2320 0.0000 0.0230 0.0005
MXNDArraySetGradState 6250 3.1470 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1250 0.5180 0.0000 0.0010 0.0004
MXAutogradBackwardEx 625 339.6850 0.5250 0.6810 0.5435
MXNDArrayGetContext 9790 4.2350 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20655 524.7000 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2500 6128.9570 0.7010 3.7500 2.4516
FullyConnected 4230 409.5110 0.0080 0.3430 0.0968
argmax 1410 27.9280 0.0050 0.0480 0.0198
_backward_pick 1250 19.7710 0.0130 0.0320 0.0158
_backward_mul_scalar 1250 5.4270 0.0030 0.0160 0.0043
DeleteVariable 52880 372.9340 0.0010 0.9010 0.0071
Flatten 1410 58.4250 0.0080 0.1120 0.0414
Activation 5640 262.6240 0.0030 1.9440 0.0466
SetValueOp 1250 8.0550 0.0050 0.0180 0.0064
pick 1250 9.8410 0.0060 0.0280 0.0079
Pooling 2820 3967.0811 0.0640 3.1720 1.4068
Convolution 2820 16058.7676 0.2260 13.2050 5.6946
CopyCPU2CPU 1410 7.9930 0.0030 0.0360 0.0057
log_softmax 1250 38.9230 0.0150 0.0550 0.0311
_backward_copy 1250 64.5240 0.0220 0.0760 0.0516
_backward_Convolution 2500 23157.4746 2.6370 12.5610 9.2630
_mul_scalar 1250 4.4800 0.0020 0.0230 0.0036
_backward_log_softmax 1250 21.2560 0.0090 0.0400 0.0170
mean 3910 75.1620 0.0080 0.0730 0.0192
_backward_mean 1250 13.7140 0.0080 0.1030 0.0110
_backward_FullyConnected 3750 714.7220 0.0280 0.5940 0.1906
WaitForVar 2660 23.7560 0.0060 0.0280 0.0089
broadcast_equal 1410 7.0490 0.0030 0.0110 0.0050
sgd_update 12500 123.4930 0.0020 0.0790 0.0099
_backward_Activation 5000 370.1030 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48724 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4236 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 625 2462.0911 0.3060 73.5850 3.9393
MXNDArrayCreateEx 706 24.1120 0.0310 0.0640 0.0342
MXNDArrayWaitAll 626 13605.1309 5.6130 25.0610 21.7334
MXNDArraySyncCopyToCPU 1332 10494.0430 0.1180 24.1980 7.8784
MXNDArrayGetDType 1332 0.4800 0.0000 0.0010 0.0004
MXNDArrayGetShape 3370 1.5730 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1412 12.3740 0.0070 0.0280 0.0088
MXNDArrayFree 15838 161.4510 0.0000 0.9140 0.0102
MXNet C API Calls 71382 558.7390 487.3580 558.7390 35.6905
MXNet C API Concurrency 142763 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1252 0.5890 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6260 3.2380 0.0000 0.0230 0.0005
MXNDArraySetGradState 6260 3.1510 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1252 0.5190 0.0000 0.0010 0.0004
MXAutogradBackwardEx 626 340.2290 0.5250 0.6810 0.5435
MXNDArrayGetContext 9804 4.2410 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20686 525.4950 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2504 6138.5278 0.7010 3.7500 2.4515
FullyConnected 4236 410.0630 0.0080 0.3430 0.0968
argmax 1412 27.9670 0.0050 0.0480 0.0198
_backward_pick 1252 19.8010 0.0130 0.0320 0.0158
_backward_mul_scalar 1252 5.4360 0.0030 0.0160 0.0043
DeleteVariable 52960 373.4720 0.0010 0.9010 0.0071
Flatten 1412 58.5070 0.0080 0.1120 0.0414
Activation 5648 262.9810 0.0030 1.9440 0.0466
SetValueOp 1252 8.0680 0.0050 0.0180 0.0064
pick 1252 9.8560 0.0060 0.0280 0.0079
Pooling 2824 3972.4651 0.0640 3.1720 1.4067
Convolution 2824 16081.0303 0.2260 13.2050 5.6944
CopyCPU2CPU 1412 8.0040 0.0030 0.0360 0.0057
log_softmax 1252 38.9820 0.0150 0.0550 0.0311
_backward_copy 1252 64.6280 0.0220 0.0760 0.0516
_backward_Convolution 2504 23194.5625 2.6370 12.5610 9.2630
_mul_scalar 1252 4.4870 0.0020 0.0230 0.0036
_backward_log_softmax 1252 21.2880 0.0090 0.0400 0.0170
mean 3916 75.2770 0.0080 0.0730 0.0192
_backward_mean 1252 13.7350 0.0080 0.1030 0.0110
_backward_FullyConnected 3756 715.7920 0.0280 0.5940 0.1906
WaitForVar 2664 23.7910 0.0060 0.0280 0.0089
broadcast_equal 1412 7.0590 0.0030 0.0110 0.0050
sgd_update 12520 123.6980 0.0020 0.0790 0.0099
_backward_Activation 5008 370.6580 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48798 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4242 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 626 2466.0581 0.3060 73.5850 3.9394
MXNDArrayCreateEx 707 24.1580 0.0310 0.0640 0.0342
MXNDArrayWaitAll 627 13626.8906 5.6130 25.0610 21.7335
MXNDArraySyncCopyToCPU 1334 10508.7881 0.1180 24.1980 7.8777
MXNDArrayGetDType 1334 0.4800 0.0000 0.0010 0.0004
MXNDArrayGetShape 3375 1.5770 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1414 12.3920 0.0070 0.0280 0.0088
MXNDArrayFree 15861 161.6980 0.0000 0.9140 0.0102
MXNet C API Calls 71489 558.8460 487.3580 558.8460 35.7440
MXNet C API Concurrency 142977 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1254 0.5900 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6270 3.2460 0.0000 0.0230 0.0005
MXNDArraySetGradState 6270 3.1570 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1254 0.5210 0.0000 0.0010 0.0004
MXAutogradBackwardEx 627 341.1740 0.5250 0.9450 0.5441
MXNDArrayGetContext 9818 4.2470 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20717 526.6710 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2508 6148.2778 0.7010 3.7500 2.4515
FullyConnected 4242 410.6740 0.0080 0.3430 0.0968
argmax 1414 28.0130 0.0050 0.0480 0.0198
_backward_pick 1254 19.8380 0.0130 0.0320 0.0158
_backward_mul_scalar 1254 5.4450 0.0030 0.0160 0.0043
DeleteVariable 53040 374.0330 0.0010 0.9010 0.0071
Flatten 1414 58.5910 0.0080 0.1120 0.0414
Activation 5656 263.3480 0.0030 1.9440 0.0466
SetValueOp 1254 8.0830 0.0050 0.0180 0.0064
pick 1254 9.8730 0.0060 0.0280 0.0079
Pooling 2828 3977.8799 0.0640 3.1720 1.4066
Convolution 2828 16107.0684 0.2260 13.2050 5.6956
CopyCPU2CPU 1414 8.0140 0.0030 0.0360 0.0057
log_softmax 1254 39.0430 0.0150 0.0550 0.0311
_backward_copy 1254 64.7270 0.0220 0.0760 0.0516
_backward_Convolution 2508 23231.4785 2.6370 12.5610 9.2629
_mul_scalar 1254 4.4950 0.0020 0.0230 0.0036
_backward_log_softmax 1254 21.3310 0.0090 0.0400 0.0170
mean 3922 75.3850 0.0080 0.0730 0.0192
_backward_mean 1254 13.7580 0.0080 0.1030 0.0110
_backward_FullyConnected 3762 717.2630 0.0280 0.5940 0.1907
WaitForVar 2668 23.8270 0.0060 0.0280 0.0089
broadcast_equal 1414 7.0690 0.0030 0.0110 0.0050
sgd_update 12540 123.9240 0.0020 0.0790 0.0099
_backward_Activation 5016 371.2370 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48872 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4248 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 627 2470.0061 0.3060 73.5850 3.9394
MXNDArrayCreateEx 708 24.2040 0.0310 0.0640 0.0342
MXNDArrayWaitAll 628 13648.5713 5.6130 25.0610 21.7334
MXNDArraySyncCopyToCPU 1336 10523.5283 0.1180 24.1980 7.8769
MXNDArrayGetDType 1336 0.4810 0.0000 0.0010 0.0004
MXNDArrayGetShape 3380 1.5800 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1416 12.4090 0.0070 0.0280 0.0088
MXNDArrayFree 15884 161.9450 0.0000 0.9140 0.0102
MXNet C API Calls 71596 558.9530 487.3580 558.9530 35.7975
MXNet C API Concurrency 143191 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1256 0.5910 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6280 3.2540 0.0000 0.0230 0.0005
MXNDArraySetGradState 6280 3.1640 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1256 0.5230 0.0000 0.0010 0.0004
MXAutogradBackwardEx 628 342.1040 0.5250 0.9450 0.5448
MXNDArrayGetContext 9832 4.2550 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20748 527.8260 0.0090 0.1450 0.0254
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2512 6157.9351 0.7010 3.7500 2.4514
FullyConnected 4248 411.2820 0.0080 0.3430 0.0968
argmax 1416 28.0550 0.0050 0.0480 0.0198
_backward_pick 1256 19.8710 0.0130 0.0320 0.0158
_backward_mul_scalar 1256 5.4540 0.0030 0.0160 0.0043
DeleteVariable 53120 374.5760 0.0010 0.9010 0.0071
Flatten 1416 58.6710 0.0080 0.1120 0.0414
Activation 5664 263.7210 0.0030 1.9440 0.0466
SetValueOp 1256 8.0980 0.0050 0.0180 0.0064
pick 1256 9.8880 0.0060 0.0280 0.0079
Pooling 2832 3983.2830 0.0640 3.1720 1.4065
Convolution 2832 16132.9912 0.2260 13.2050 5.6967
CopyCPU2CPU 1416 8.0240 0.0030 0.0360 0.0057
log_softmax 1256 39.1150 0.0150 0.0550 0.0311
_backward_copy 1256 64.8290 0.0220 0.0760 0.0516
_backward_Convolution 2512 23268.2988 2.6370 12.5610 9.2629
_mul_scalar 1256 4.5020 0.0020 0.0230 0.0036
_backward_log_softmax 1256 21.3640 0.0090 0.0400 0.0170
mean 3928 75.4920 0.0080 0.0730 0.0192
_backward_mean 1256 13.7830 0.0080 0.1030 0.0110
_backward_FullyConnected 3768 718.7420 0.0280 0.5940 0.1907
WaitForVar 2672 23.8630 0.0060 0.0280 0.0089
broadcast_equal 1416 7.0790 0.0030 0.0110 0.0050
sgd_update 12560 124.1270 0.0020 0.0790 0.0099
_backward_Activation 5024 371.8630 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 48946 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4254 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 628 2473.9441 0.3060 73.5850 3.9394
MXNDArrayCreateEx 709 24.2490 0.0310 0.0640 0.0342
MXNDArrayWaitAll 629 13670.4170 5.6130 25.0610 21.7336
MXNDArraySyncCopyToCPU 1338 10538.1895 0.1180 24.1980 7.8761
MXNDArrayGetDType 1338 0.4830 0.0000 0.0010 0.0004
MXNDArrayGetShape 3385 1.5840 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1418 12.4260 0.0070 0.0280 0.0088
MXNDArrayFree 15907 162.1930 0.0000 0.9140 0.0102
MXNet C API Calls 71703 559.0600 487.3580 559.0600 35.8510
MXNet C API Concurrency 143405 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1258 0.5910 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6290 3.2600 0.0000 0.0230 0.0005
MXNDArraySetGradState 6290 3.1720 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1258 0.5230 0.0000 0.0010 0.0004
MXAutogradBackwardEx 629 343.0220 0.5250 0.9450 0.5453
MXNDArrayGetContext 9846 4.2630 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20779 528.9830 0.0090 0.1450 0.0255
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2516 6167.6641 0.7010 3.7500 2.4514
FullyConnected 4254 411.8110 0.0080 0.3430 0.0968
argmax 1418 28.0970 0.0050 0.0480 0.0198
_backward_pick 1258 19.9030 0.0130 0.0320 0.0158
_backward_mul_scalar 1258 5.4630 0.0030 0.0160 0.0043
DeleteVariable 53200 375.1230 0.0010 0.9010 0.0071
Flatten 1418 58.7510 0.0080 0.1120 0.0414
Activation 5672 264.0760 0.0030 1.9440 0.0466
SetValueOp 1258 8.1120 0.0050 0.0180 0.0064
pick 1258 9.9030 0.0060 0.0280 0.0079
Pooling 2836 3988.6550 0.0640 3.1720 1.4064
Convolution 2836 16158.9873 0.2260 13.2050 5.6978
CopyCPU2CPU 1418 8.0340 0.0030 0.0360 0.0057
log_softmax 1258 39.1760 0.0150 0.0550 0.0311
_backward_copy 1258 64.9300 0.0220 0.0760 0.0516
_backward_Convolution 2516 23305.3535 2.6370 12.5610 9.2629
_mul_scalar 1258 4.5100 0.0020 0.0230 0.0036
_backward_log_softmax 1258 21.3960 0.0090 0.0400 0.0170
mean 3934 75.6090 0.0080 0.0730 0.0192
_backward_mean 1258 13.8050 0.0080 0.1030 0.0110
_backward_FullyConnected 3774 720.2050 0.0280 0.5940 0.1908
WaitForVar 2676 23.8990 0.0060 0.0280 0.0089
broadcast_equal 1418 7.0890 0.0030 0.0110 0.0050
sgd_update 12580 124.3480 0.0020 0.0790 0.0099
_backward_Activation 5032 372.4340 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49020 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4260 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 629 2477.8931 0.3060 73.5850 3.9394
MXNDArrayCreateEx 710 24.2950 0.0310 0.0640 0.0342
MXNDArrayWaitAll 630 13692.2656 5.6130 25.0610 21.7338
MXNDArraySyncCopyToCPU 1340 10552.9102 0.1180 24.1980 7.8753
MXNDArrayGetDType 1340 0.4830 0.0000 0.0010 0.0004
MXNDArrayGetShape 3390 1.5880 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1420 12.4430 0.0070 0.0280 0.0088
MXNDArrayFree 15930 162.4300 0.0000 0.9140 0.0102
MXNet C API Calls 71810 559.1670 487.3580 559.1670 35.9045
MXNet C API Concurrency 143619 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1260 0.5910 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6300 3.2660 0.0000 0.0230 0.0005
MXNDArraySetGradState 6300 3.1780 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1260 0.5230 0.0000 0.0010 0.0004
MXAutogradBackwardEx 630 343.9410 0.5250 0.9450 0.5459
MXNDArrayGetContext 9860 4.2710 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20810 530.1540 0.0090 0.1450 0.0255
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2520 6177.3179 0.7010 3.7500 2.4513
FullyConnected 4260 412.3350 0.0080 0.3430 0.0968
argmax 1420 28.1390 0.0050 0.0480 0.0198
_backward_pick 1260 19.9340 0.0130 0.0320 0.0158
_backward_mul_scalar 1260 5.4720 0.0030 0.0160 0.0043
DeleteVariable 53280 375.6630 0.0010 0.9010 0.0071
Flatten 1420 58.8330 0.0080 0.1120 0.0414
Activation 5680 264.4300 0.0030 1.9440 0.0466
SetValueOp 1260 8.1260 0.0050 0.0180 0.0064
pick 1260 9.9180 0.0060 0.0280 0.0079
Pooling 2840 3994.0720 0.0640 3.1720 1.4064
Convolution 2840 16185.0215 0.2260 13.2050 5.6990
CopyCPU2CPU 1420 8.0450 0.0030 0.0360 0.0057
log_softmax 1260 39.2370 0.0150 0.0550 0.0311
_backward_copy 1260 65.0320 0.0220 0.0760 0.0516
_backward_Convolution 2520 23342.5293 2.6370 12.5610 9.2629
_mul_scalar 1260 4.5170 0.0020 0.0230 0.0036
_backward_log_softmax 1260 21.4280 0.0090 0.0400 0.0170
mean 3940 75.7320 0.0080 0.0730 0.0192
_backward_mean 1260 13.8260 0.0080 0.1030 0.0110
_backward_FullyConnected 3780 721.6610 0.0280 0.5940 0.1909
WaitForVar 2680 23.9360 0.0060 0.0280 0.0089
broadcast_equal 1420 7.0980 0.0030 0.0110 0.0050
sgd_update 12600 124.5670 0.0020 0.0790 0.0099
_backward_Activation 5040 372.9910 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49094 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4266 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 630 2481.8391 0.3060 73.5850 3.9394
MXNDArrayCreateEx 711 24.3410 0.0310 0.0640 0.0342
MXNDArrayWaitAll 631 13713.9463 5.6130 25.0610 21.7337
MXNDArraySyncCopyToCPU 1342 10567.6484 0.1180 24.1980 7.8746
MXNDArrayGetDType 1342 0.4840 0.0000 0.0010 0.0004
MXNDArrayGetShape 3395 1.5920 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1422 12.4600 0.0070 0.0280 0.0088
MXNDArrayFree 15953 162.6860 0.0000 0.9140 0.0102
MXNet C API Calls 71917 559.2740 487.3580 559.2740 35.9580
MXNet C API Concurrency 143833 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1262 0.5930 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6310 3.2740 0.0000 0.0230 0.0005
MXNDArraySetGradState 6310 3.1840 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1262 0.5250 0.0000 0.0010 0.0004
MXAutogradBackwardEx 631 344.8780 0.5250 0.9450 0.5466
MXNDArrayGetContext 9874 4.2810 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20841 531.3120 0.0090 0.1450 0.0255
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2524 6186.9009 0.7010 3.7500 2.4512
FullyConnected 4266 412.9610 0.0080 0.3430 0.0968
argmax 1422 28.1860 0.0050 0.0480 0.0198
_backward_pick 1262 19.9650 0.0130 0.0320 0.0158
_backward_mul_scalar 1262 5.4810 0.0030 0.0160 0.0043
DeleteVariable 53360 376.2500 0.0010 0.9010 0.0071
Flatten 1422 58.9140 0.0080 0.1120 0.0414
Activation 5688 264.8020 0.0030 1.9440 0.0466
SetValueOp 1262 8.1410 0.0050 0.0180 0.0065
pick 1262 9.9330 0.0060 0.0280 0.0079
Pooling 2844 3999.4441 0.0640 3.1720 1.4063
Convolution 2844 16211.0088 0.2260 13.2050 5.7001
CopyCPU2CPU 1422 8.0550 0.0030 0.0360 0.0057
log_softmax 1262 39.2970 0.0150 0.0550 0.0311
_backward_copy 1262 65.1320 0.0220 0.0760 0.0516
_backward_Convolution 2524 23379.3809 2.6370 12.5610 9.2628
_mul_scalar 1262 4.5230 0.0020 0.0230 0.0036
_backward_log_softmax 1262 21.4700 0.0090 0.0400 0.0170
mean 3946 75.8380 0.0080 0.0730 0.0192
_backward_mean 1262 13.8490 0.0080 0.1030 0.0110
_backward_FullyConnected 3786 723.1420 0.0280 0.5940 0.1910
WaitForVar 2684 23.9720 0.0060 0.0280 0.0089
broadcast_equal 1422 7.1070 0.0030 0.0110 0.0050
sgd_update 12620 124.7910 0.0020 0.0790 0.0099
_backward_Activation 5048 373.5690 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49168 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4272 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 631 2485.7649 0.3060 73.5850 3.9394
MXNDArrayCreateEx 712 24.3860 0.0310 0.0640 0.0342
MXNDArrayWaitAll 632 13735.6992 5.6130 25.0610 21.7337
MXNDArraySyncCopyToCPU 1344 10582.3145 0.1180 24.1980 7.8737
MXNDArrayGetDType 1344 0.4840 0.0000 0.0010 0.0004
MXNDArrayGetShape 3400 1.5930 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1424 12.4770 0.0070 0.0280 0.0088
MXNDArrayFree 15976 162.9330 0.0000 0.9140 0.0102
MXNet C API Calls 72024 559.3810 487.3580 559.3810 36.0115
MXNet C API Concurrency 144047 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1264 0.5940 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6320 3.2790 0.0000 0.0230 0.0005
MXNDArraySetGradState 6320 3.1930 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1264 0.5270 0.0000 0.0010 0.0004
MXAutogradBackwardEx 632 345.7960 0.5250 0.9450 0.5471
MXNDArrayGetContext 9888 4.2880 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20872 532.4680 0.0090 0.1450 0.0255
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2528 6196.5581 0.7010 3.7500 2.4512
FullyConnected 4272 413.5280 0.0080 0.3430 0.0968
argmax 1424 28.2330 0.0050 0.0480 0.0198
_backward_pick 1264 20.0000 0.0130 0.0320 0.0158
_backward_mul_scalar 1264 5.4900 0.0030 0.0160 0.0043
DeleteVariable 53440 376.7990 0.0010 0.9010 0.0071
Flatten 1424 58.9990 0.0080 0.1120 0.0414
Activation 5696 265.1710 0.0030 1.9440 0.0466
SetValueOp 1264 8.1550 0.0050 0.0180 0.0065
pick 1264 9.9500 0.0060 0.0280 0.0079
Pooling 2848 4004.8689 0.0640 3.1720 1.4062
Convolution 2848 16236.9180 0.2260 13.2050 5.7012
CopyCPU2CPU 1424 8.0650 0.0030 0.0360 0.0057
log_softmax 1264 39.3570 0.0150 0.0550 0.0311
_backward_copy 1264 65.2320 0.0220 0.0760 0.0516
_backward_Convolution 2528 23416.3027 2.6370 12.5610 9.2628
_mul_scalar 1264 4.5300 0.0020 0.0230 0.0036
_backward_log_softmax 1264 21.5120 0.0090 0.0400 0.0170
mean 3952 75.9450 0.0080 0.0730 0.0192
_backward_mean 1264 13.8720 0.0080 0.1030 0.0110
_backward_FullyConnected 3792 724.6100 0.0280 0.5940 0.1911
WaitForVar 2688 24.0080 0.0060 0.0280 0.0089
broadcast_equal 1424 7.1160 0.0030 0.0110 0.0050
sgd_update 12640 125.0230 0.0020 0.0790 0.0099
_backward_Activation 5056 374.1380 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49242 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4278 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 632 2489.6880 0.3060 73.5850 3.9394
MXNDArrayCreateEx 713 24.4310 0.0310 0.0640 0.0343
MXNDArrayWaitAll 633 13757.5791 5.6130 25.0610 21.7339
MXNDArraySyncCopyToCPU 1346 10597.0039 0.1180 24.1980 7.8730
MXNDArrayGetDType 1346 0.4860 0.0000 0.0010 0.0004
MXNDArrayGetShape 3405 1.5940 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1426 12.4940 0.0070 0.0280 0.0088
MXNDArrayFree 15999 163.1810 0.0000 0.9140 0.0102
MXNet C API Calls 72131 559.4880 487.3580 559.4880 36.0650
MXNet C API Concurrency 144261 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1266 0.5960 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6330 3.2870 0.0000 0.0230 0.0005
MXNDArraySetGradState 6330 3.2000 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1266 0.5280 0.0000 0.0010 0.0004
MXAutogradBackwardEx 633 346.6860 0.5250 0.9450 0.5477
MXNDArrayGetContext 9902 4.2970 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20903 533.6300 0.0090 0.1450 0.0255
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2532 6206.3418 0.7010 3.7500 2.4512
FullyConnected 4278 414.0520 0.0080 0.3430 0.0968
argmax 1426 28.2760 0.0050 0.0480 0.0198
_backward_pick 1266 20.0330 0.0130 0.0320 0.0158
_backward_mul_scalar 1266 5.4990 0.0030 0.0160 0.0043
DeleteVariable 53520 377.3430 0.0010 0.9010 0.0071
Flatten 1426 59.0800 0.0080 0.1120 0.0414
Activation 5704 265.5610 0.0030 1.9440 0.0466
SetValueOp 1266 8.1700 0.0050 0.0180 0.0065
pick 1266 9.9660 0.0060 0.0280 0.0079
Pooling 2852 4010.2549 0.0640 3.1720 1.4061
Convolution 2852 16262.8281 0.2260 13.2050 5.7023
CopyCPU2CPU 1426 8.0750 0.0030 0.0360 0.0057
log_softmax 1266 39.4160 0.0150 0.0550 0.0311
_backward_copy 1266 65.3330 0.0220 0.0760 0.0516
_backward_Convolution 2532 23453.4004 2.6370 12.5610 9.2628
_mul_scalar 1266 4.5370 0.0020 0.0230 0.0036
_backward_log_softmax 1266 21.5450 0.0090 0.0400 0.0170
mean 3958 76.0620 0.0080 0.0730 0.0192
_backward_mean 1266 13.8940 0.0080 0.1030 0.0110
_backward_FullyConnected 3798 726.0690 0.0280 0.5940 0.1912
WaitForVar 2692 24.0500 0.0060 0.0280 0.0089
broadcast_equal 1426 7.1260 0.0030 0.0110 0.0050
sgd_update 12660 125.1980 0.0020 0.0790 0.0099
_backward_Activation 5064 374.7690 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49316 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4284 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 633 2493.5950 0.3060 73.5850 3.9393
MXNDArrayCreateEx 714 24.4770 0.0310 0.0640 0.0343
MXNDArrayWaitAll 634 13779.5166 5.6130 25.0610 21.7343
MXNDArraySyncCopyToCPU 1348 10611.7578 0.1180 24.1980 7.8722
MXNDArrayGetDType 1348 0.4880 0.0000 0.0010 0.0004
MXNDArrayGetShape 3410 1.5970 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1428 12.5130 0.0070 0.0280 0.0088
MXNDArrayFree 16022 163.4260 0.0000 0.9140 0.0102
MXNet C API Calls 72238 559.5950 487.3580 559.5950 36.1185
MXNet C API Concurrency 144475 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1268 0.5970 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6340 3.2950 0.0000 0.0230 0.0005
MXNDArraySetGradState 6340 3.2040 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1268 0.5300 0.0000 0.0010 0.0004
MXAutogradBackwardEx 634 347.5700 0.5250 0.9450 0.5482
MXNDArrayGetContext 9916 4.3080 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20934 534.7880 0.0090 0.1450 0.0255
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2536 6216.1348 0.7010 3.7500 2.4512
FullyConnected 4284 414.6250 0.0080 0.3430 0.0968
argmax 1428 28.3180 0.0050 0.0480 0.0198
_backward_pick 1268 20.0640 0.0130 0.0320 0.0158
_backward_mul_scalar 1268 5.5080 0.0030 0.0160 0.0043
DeleteVariable 53600 377.8930 0.0010 0.9010 0.0071
Flatten 1428 59.1790 0.0080 0.1120 0.0414
Activation 5712 265.9550 0.0030 1.9440 0.0466
SetValueOp 1268 8.1850 0.0050 0.0180 0.0065
pick 1268 9.9810 0.0060 0.0280 0.0079
Pooling 2856 4015.6450 0.0640 3.1720 1.4060
Convolution 2856 16288.7676 0.2260 13.2050 5.7034
CopyCPU2CPU 1428 8.0840 0.0030 0.0360 0.0057
log_softmax 1268 39.4750 0.0150 0.0550 0.0311
_backward_copy 1268 65.4340 0.0220 0.0760 0.0516
_backward_Convolution 2536 23490.6094 2.6370 12.5610 9.2629
_mul_scalar 1268 4.5440 0.0020 0.0230 0.0036
_backward_log_softmax 1268 21.5770 0.0090 0.0400 0.0170
mean 3964 76.1830 0.0080 0.0730 0.0192
_backward_mean 1268 13.9160 0.0080 0.1030 0.0110
_backward_FullyConnected 3804 727.5260 0.0280 0.5940 0.1913
WaitForVar 2696 24.0840 0.0060 0.0280 0.0089
broadcast_equal 1428 7.1370 0.0030 0.0110 0.0050
sgd_update 12680 125.4240 0.0020 0.0790 0.0099
_backward_Activation 5072 375.3250 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49390 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4290 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 634 2493.9221 0.3060 73.5850 3.9336
MXNDArrayCreateEx 715 24.5220 0.0310 0.0640 0.0343
MXNDArrayWaitAll 635 13801.2432 5.6130 25.0610 21.7342
MXNDArraySyncCopyToCPU 1350 10626.5332 0.1180 24.1980 7.8715
MXNDArrayGetDType 1350 0.4890 0.0000 0.0010 0.0004
MXNDArrayGetShape 3415 1.6020 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1430 12.5310 0.0070 0.0280 0.0088
MXNDArrayFree 16045 163.6780 0.0000 0.9140 0.0102
MXNet C API Calls 72345 559.7020 487.3580 559.7020 36.1720
MXNet C API Concurrency 144689 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1270 0.5990 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6350 3.3010 0.0000 0.0230 0.0005
MXNDArraySetGradState 6350 3.2120 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1270 0.5310 0.0000 0.0010 0.0004
MXAutogradBackwardEx 635 348.4900 0.5250 0.9450 0.5488
MXNDArrayGetContext 9930 4.3140 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20965 535.9370 0.0090 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2540 6225.9312 0.7010 3.7500 2.4512
FullyConnected 4290 415.2230 0.0080 0.3430 0.0968
argmax 1430 28.3620 0.0050 0.0480 0.0198
_backward_pick 1270 20.0970 0.0130 0.0320 0.0158
_backward_mul_scalar 1270 5.5170 0.0030 0.0160 0.0043
DeleteVariable 53680 378.4560 0.0010 0.9010 0.0071
Flatten 1430 59.2600 0.0080 0.1120 0.0414
Activation 5720 266.3240 0.0030 1.9440 0.0466
SetValueOp 1270 8.2000 0.0050 0.0180 0.0065
pick 1270 9.9960 0.0060 0.0280 0.0079
Pooling 2860 4021.0549 0.0640 3.1720 1.4060
Convolution 2860 16314.7461 0.2260 13.2050 5.7045
CopyCPU2CPU 1430 8.0940 0.0030 0.0360 0.0057
log_softmax 1270 39.5490 0.0150 0.0550 0.0311
_backward_copy 1270 65.5340 0.0220 0.0760 0.0516
_backward_Convolution 2540 23527.4121 2.6370 12.5610 9.2628
_mul_scalar 1270 4.5500 0.0020 0.0230 0.0036
_backward_log_softmax 1270 21.6090 0.0090 0.0400 0.0170
mean 3970 76.2910 0.0080 0.0730 0.0192
_backward_mean 1270 13.9390 0.0080 0.1030 0.0110
_backward_FullyConnected 3810 729.0020 0.0280 0.5940 0.1913
WaitForVar 2700 24.1200 0.0060 0.0280 0.0089
broadcast_equal 1430 7.1470 0.0030 0.0110 0.0050
sgd_update 12700 125.6260 0.0020 0.0790 0.0099
_backward_Activation 5080 375.9500 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49464 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4296 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 635 2497.8220 0.3060 73.5850 3.9336
MXNDArrayCreateEx 716 24.5690 0.0310 0.0640 0.0343
MXNDArrayWaitAll 636 13822.9863 5.6130 25.0610 21.7343
MXNDArraySyncCopyToCPU 1352 10641.2598 0.1180 24.1980 7.8708
MXNDArrayGetDType 1352 0.4890 0.0000 0.0010 0.0004
MXNDArrayGetShape 3420 1.6040 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1432 12.5490 0.0070 0.0280 0.0088
MXNDArrayFree 16068 163.9220 0.0000 0.9140 0.0102
MXNet C API Calls 72452 559.8090 487.3580 559.8090 36.2255
MXNet C API Concurrency 144903 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1272 0.6000 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6360 3.3080 0.0000 0.0230 0.0005
MXNDArraySetGradState 6360 3.2200 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1272 0.5310 0.0000 0.0010 0.0004
MXAutogradBackwardEx 636 349.4040 0.5250 0.9450 0.5494
MXNDArrayGetContext 9944 4.3240 0.0000 0.0070 0.0004
MXImperativeInvokeEx 20996 537.1030 0.0090 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2544 6235.6958 0.7010 3.7500 2.4511
FullyConnected 4296 415.8090 0.0080 0.3430 0.0968
argmax 1432 28.4040 0.0050 0.0480 0.0198
_backward_pick 1272 20.1310 0.0130 0.0320 0.0158
_backward_mul_scalar 1272 5.5260 0.0030 0.0160 0.0043
DeleteVariable 53760 379.0090 0.0010 0.9010 0.0071
Flatten 1432 59.3430 0.0080 0.1120 0.0414
Activation 5728 266.6950 0.0030 1.9440 0.0466
SetValueOp 1272 8.2140 0.0050 0.0180 0.0065
pick 1272 10.0130 0.0060 0.0280 0.0079
Pooling 2864 4026.4609 0.0640 3.1720 1.4059
Convolution 2864 16340.6650 0.2260 13.2050 5.7055
CopyCPU2CPU 1432 8.1040 0.0030 0.0360 0.0057
log_softmax 1272 39.6230 0.0150 0.0550 0.0312
_backward_copy 1272 65.6330 0.0220 0.0760 0.0516
_backward_Convolution 2544 23564.2363 2.6370 12.5610 9.2627
_mul_scalar 1272 4.5570 0.0020 0.0230 0.0036
_backward_log_softmax 1272 21.6400 0.0090 0.0400 0.0170
mean 3976 76.4000 0.0080 0.0730 0.0192
_backward_mean 1272 13.9610 0.0080 0.1030 0.0110
_backward_FullyConnected 3816 730.4900 0.0280 0.5940 0.1914
WaitForVar 2704 24.1570 0.0060 0.0280 0.0089
broadcast_equal 1432 7.1570 0.0030 0.0110 0.0050
sgd_update 12720 125.8560 0.0020 0.0790 0.0099
_backward_Activation 5088 376.5160 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49538 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4302 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 636 2501.7271 0.3060 73.5850 3.9335
MXNDArrayCreateEx 717 24.6160 0.0310 0.0640 0.0343
MXNDArrayWaitAll 637 13844.8857 5.6130 25.0610 21.7345
MXNDArraySyncCopyToCPU 1354 10655.9199 0.1180 24.1980 7.8700
MXNDArrayGetDType 1354 0.4910 0.0000 0.0010 0.0004
MXNDArrayGetShape 3425 1.6070 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1434 12.5670 0.0070 0.0280 0.0088
MXNDArrayFree 16091 164.1760 0.0000 0.9140 0.0102
MXNet C API Calls 72559 559.9160 487.3580 559.9160 36.2790
MXNet C API Concurrency 145117 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1274 0.6010 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6370 3.3150 0.0000 0.0230 0.0005
MXNDArraySetGradState 6370 3.2290 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1274 0.5330 0.0000 0.0010 0.0004
MXAutogradBackwardEx 637 350.3190 0.5250 0.9450 0.5500
MXNDArrayGetContext 9958 4.3310 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21027 538.2640 0.0090 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2548 6245.4121 0.7010 3.7500 2.4511
FullyConnected 4302 416.3610 0.0080 0.3430 0.0968
argmax 1434 28.4460 0.0050 0.0480 0.0198
_backward_pick 1274 20.1640 0.0130 0.0320 0.0158
_backward_mul_scalar 1274 5.5350 0.0030 0.0160 0.0043
DeleteVariable 53840 379.5620 0.0010 0.9010 0.0070
Flatten 1434 59.4280 0.0080 0.1120 0.0414
Activation 5736 267.0690 0.0030 1.9440 0.0466
SetValueOp 1274 8.2290 0.0050 0.0180 0.0065
pick 1274 10.0290 0.0060 0.0280 0.0079
Pooling 2868 4031.8640 0.0640 3.1720 1.4058
Convolution 2868 16366.6035 0.2260 13.2050 5.7066
CopyCPU2CPU 1434 8.1130 0.0030 0.0360 0.0057
log_softmax 1274 39.6830 0.0150 0.0550 0.0311
_backward_copy 1274 65.7340 0.0220 0.0760 0.0516
_backward_Convolution 2548 23601.4180 2.6370 12.5610 9.2627
_mul_scalar 1274 4.5640 0.0020 0.0230 0.0036
_backward_log_softmax 1274 21.6720 0.0090 0.0400 0.0170
mean 3982 76.5150 0.0080 0.0730 0.0192
_backward_mean 1274 13.9830 0.0080 0.1030 0.0110
_backward_FullyConnected 3822 731.9520 0.0280 0.5940 0.1915
WaitForVar 2708 24.1940 0.0060 0.0280 0.0089
broadcast_equal 1434 7.1670 0.0030 0.0110 0.0050
sgd_update 12740 126.0280 0.0020 0.0790 0.0099
_backward_Activation 5096 377.1370 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49612 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4308 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 637 2505.6560 0.3060 73.5850 3.9335
MXNDArrayCreateEx 718 24.6610 0.0310 0.0640 0.0343
MXNDArrayWaitAll 638 13866.7744 5.6130 25.0610 21.7348
MXNDArraySyncCopyToCPU 1356 10670.6387 0.1180 24.1980 7.8692
MXNDArrayGetDType 1356 0.4910 0.0000 0.0010 0.0004
MXNDArrayGetShape 3430 1.6080 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1436 12.5830 0.0070 0.0280 0.0088
MXNDArrayFree 16114 164.4240 0.0000 0.9140 0.0102
MXNet C API Calls 72666 560.0230 487.3580 560.0230 36.3325
MXNet C API Concurrency 145331 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1276 0.6030 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6380 3.3220 0.0000 0.0230 0.0005
MXNDArraySetGradState 6380 3.2360 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1276 0.5340 0.0000 0.0010 0.0004
MXAutogradBackwardEx 638 351.2330 0.5250 0.9450 0.5505
MXNDArrayGetContext 9972 4.3420 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21058 539.4180 0.0090 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2552 6255.2021 0.7010 3.7500 2.4511
FullyConnected 4308 416.8960 0.0080 0.3430 0.0968
argmax 1436 28.4870 0.0050 0.0480 0.0198
_backward_pick 1276 20.1960 0.0130 0.0320 0.0158
_backward_mul_scalar 1276 5.5440 0.0030 0.0160 0.0043
DeleteVariable 53920 380.1140 0.0010 0.9010 0.0070
Flatten 1436 59.5230 0.0080 0.1120 0.0415
Activation 5744 267.4570 0.0030 1.9440 0.0466
SetValueOp 1276 8.2430 0.0050 0.0180 0.0065
pick 1276 10.0450 0.0060 0.0280 0.0079
Pooling 2872 4037.2710 0.0640 3.1720 1.4057
Convolution 2872 16392.5527 0.2260 13.2050 5.7077
CopyCPU2CPU 1436 8.1220 0.0030 0.0360 0.0057
log_softmax 1276 39.7420 0.0150 0.0550 0.0311
_backward_copy 1276 65.8360 0.0220 0.0760 0.0516
_backward_Convolution 2552 23638.5352 2.6370 12.5610 9.2627
_mul_scalar 1276 4.5710 0.0020 0.0230 0.0036
_backward_log_softmax 1276 21.7030 0.0090 0.0400 0.0170
mean 3988 76.6380 0.0080 0.0730 0.0192
_backward_mean 1276 14.0050 0.0080 0.1030 0.0110
_backward_FullyConnected 3828 733.4100 0.0280 0.5940 0.1916
WaitForVar 2712 24.2300 0.0060 0.0280 0.0089
broadcast_equal 1436 7.1760 0.0030 0.0110 0.0050
sgd_update 12760 126.1970 0.0020 0.0790 0.0099
_backward_Activation 5104 377.7590 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49686 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4314 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 638 2509.5740 0.3060 73.5850 3.9335
MXNDArrayCreateEx 719 24.7070 0.0310 0.0640 0.0344
MXNDArrayWaitAll 639 13888.5752 5.6130 25.0610 21.7349
MXNDArraySyncCopyToCPU 1358 10685.4238 0.1180 24.1980 7.8685
MXNDArrayGetDType 1358 0.4930 0.0000 0.0010 0.0004
MXNDArrayGetShape 3435 1.6110 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1438 12.6010 0.0070 0.0280 0.0088
MXNDArrayFree 16137 164.6670 0.0000 0.9140 0.0102
MXNet C API Calls 72773 560.1300 487.3580 560.1300 36.3860
MXNet C API Concurrency 145545 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1278 0.6040 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6390 3.3300 0.0000 0.0230 0.0005
MXNDArraySetGradState 6390 3.2450 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1278 0.5350 0.0000 0.0010 0.0004
MXAutogradBackwardEx 639 352.1300 0.5250 0.9450 0.5511
MXNDArrayGetContext 9986 4.3490 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21089 540.5750 0.0090 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2556 6265.1162 0.7010 3.7500 2.4511
FullyConnected 4314 417.5110 0.0080 0.3430 0.0968
argmax 1438 28.5330 0.0050 0.0480 0.0198
_backward_pick 1278 20.2270 0.0130 0.0320 0.0158
_backward_mul_scalar 1278 5.5530 0.0030 0.0160 0.0043
DeleteVariable 54000 380.6810 0.0010 0.9010 0.0070
Flatten 1438 59.6040 0.0080 0.1120 0.0414
Activation 5752 267.8280 0.0030 1.9440 0.0466
SetValueOp 1278 8.2580 0.0050 0.0180 0.0065
pick 1278 10.0610 0.0060 0.0280 0.0079
Pooling 2876 4042.6970 0.0640 3.1720 1.4057
Convolution 2876 16418.4844 0.2260 13.2050 5.7088
CopyCPU2CPU 1438 8.1330 0.0030 0.0360 0.0057
log_softmax 1278 39.8010 0.0150 0.0550 0.0311
_backward_copy 1278 65.9370 0.0220 0.0760 0.0516
_backward_Convolution 2556 23675.4023 2.6370 12.5610 9.2627
_mul_scalar 1278 4.5790 0.0020 0.0230 0.0036
_backward_log_softmax 1278 21.7450 0.0090 0.0400 0.0170
mean 3994 76.7610 0.0080 0.0730 0.0192
_backward_mean 1278 14.0280 0.0080 0.1030 0.0110
_backward_FullyConnected 3834 734.8670 0.0280 0.5940 0.1917
WaitForVar 2716 24.2660 0.0060 0.0280 0.0089
broadcast_equal 1438 7.1860 0.0030 0.0110 0.0050
sgd_update 12780 126.4220 0.0020 0.0790 0.0099
_backward_Activation 5112 378.3210 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49760 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4320 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 639 2513.4910 0.3060 73.5850 3.9335
MXNDArrayCreateEx 720 24.7530 0.0310 0.0640 0.0344
MXNDArrayWaitAll 640 13910.3105 5.6130 25.0610 21.7349
MXNDArraySyncCopyToCPU 1360 10700.1582 0.1180 24.1980 7.8678
MXNDArrayGetDType 1360 0.4940 0.0000 0.0010 0.0004
MXNDArrayGetShape 3440 1.6130 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1440 12.6180 0.0070 0.0280 0.0088
MXNDArrayFree 16160 164.9160 0.0000 0.9140 0.0102
MXNet C API Calls 72880 560.2370 487.3580 560.2370 36.4395
MXNet C API Concurrency 145759 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1280 0.6040 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6400 3.3380 0.0000 0.0230 0.0005
MXNDArraySetGradState 6400 3.2510 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1280 0.5350 0.0000 0.0010 0.0004
MXAutogradBackwardEx 640 353.0160 0.5250 0.9450 0.5516
MXNDArrayGetContext 10000 4.3580 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21120 541.7590 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2560 6274.8408 0.7010 3.7500 2.4511
FullyConnected 4320 418.1220 0.0080 0.3430 0.0968
argmax 1440 28.5790 0.0050 0.0480 0.0198
_backward_pick 1280 20.2580 0.0130 0.0320 0.0158
_backward_mul_scalar 1280 5.5620 0.0030 0.0160 0.0043
DeleteVariable 54080 381.2540 0.0010 0.9010 0.0070
Flatten 1440 59.6870 0.0080 0.1120 0.0414
Activation 5760 268.2040 0.0030 1.9440 0.0466
SetValueOp 1280 8.2710 0.0050 0.0180 0.0065
pick 1280 10.0790 0.0060 0.0280 0.0079
Pooling 2880 4048.1709 0.0640 3.1720 1.4056
Convolution 2880 16444.3164 0.2260 13.2050 5.7098
CopyCPU2CPU 1440 8.1430 0.0030 0.0360 0.0057
log_softmax 1280 39.8610 0.0150 0.0550 0.0311
_backward_copy 1280 66.0360 0.0220 0.0760 0.0516
_backward_Convolution 2560 23712.2676 2.6370 12.5610 9.2626
_mul_scalar 1280 4.5860 0.0020 0.0230 0.0036
_backward_log_softmax 1280 21.7870 0.0090 0.0400 0.0170
mean 4000 76.8690 0.0080 0.0730 0.0192
_backward_mean 1280 14.0500 0.0080 0.1030 0.0110
_backward_FullyConnected 3840 736.3360 0.0280 0.5940 0.1918
WaitForVar 2720 24.3020 0.0060 0.0280 0.0089
broadcast_equal 1440 7.1960 0.0030 0.0110 0.0050
sgd_update 12800 126.6420 0.0020 0.0790 0.0099
_backward_Activation 5120 378.8870 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49834 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4326 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 640 2517.4309 0.3060 73.5850 3.9335
MXNDArrayCreateEx 721 24.8040 0.0310 0.0640 0.0344
MXNDArrayWaitAll 641 13932.2119 5.6130 25.0610 21.7351
MXNDArraySyncCopyToCPU 1362 10714.8770 0.1180 24.1980 7.8670
MXNDArrayGetDType 1362 0.4960 0.0000 0.0010 0.0004
MXNDArrayGetShape 3445 1.6140 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1442 12.6360 0.0070 0.0280 0.0088
MXNDArrayFree 16183 165.1660 0.0000 0.9140 0.0102
MXNet C API Calls 72987 560.3440 487.3580 560.3440 36.4930
MXNet C API Concurrency 145973 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1282 0.6050 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6410 3.3460 0.0000 0.0230 0.0005
MXNDArraySetGradState 6410 3.2570 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1282 0.5370 0.0000 0.0010 0.0004
MXAutogradBackwardEx 641 353.9090 0.5250 0.9450 0.5521
MXNDArrayGetContext 10014 4.3660 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21151 542.9130 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2564 6284.6011 0.7010 3.7500 2.4511
FullyConnected 4326 418.6510 0.0080 0.3430 0.0968
argmax 1442 28.6210 0.0050 0.0480 0.0198
_backward_pick 1282 20.2900 0.0130 0.0320 0.0158
_backward_mul_scalar 1282 5.6040 0.0030 0.0220 0.0044
DeleteVariable 54160 381.8170 0.0010 0.9010 0.0070
Flatten 1442 59.7720 0.0080 0.1120 0.0415
Activation 5768 268.5610 0.0030 1.9440 0.0466
SetValueOp 1282 8.2860 0.0050 0.0180 0.0065
pick 1282 10.0940 0.0060 0.0280 0.0079
Pooling 2884 4053.6121 0.0640 3.1720 1.4056
Convolution 2884 16470.2246 0.2260 13.2050 5.7109
CopyCPU2CPU 1442 8.1530 0.0030 0.0360 0.0057
log_softmax 1282 39.9210 0.0150 0.0550 0.0311
_backward_copy 1282 66.1370 0.0220 0.0760 0.0516
_backward_Convolution 2564 23749.3926 2.6370 12.5610 9.2626
_mul_scalar 1282 4.5930 0.0020 0.0230 0.0036
_backward_log_softmax 1282 21.8190 0.0090 0.0400 0.0170
mean 4006 77.0370 0.0080 0.0730 0.0192
_backward_mean 1282 14.0720 0.0080 0.1030 0.0110
_backward_FullyConnected 3846 737.8060 0.0280 0.5940 0.1918
WaitForVar 2724 24.3370 0.0060 0.0280 0.0089
broadcast_equal 1442 7.2060 0.0030 0.0110 0.0050
sgd_update 12820 126.8660 0.0020 0.0790 0.0099
_backward_Activation 5128 379.4610 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49908 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4332 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 641 2521.3630 0.3060 73.5850 3.9335
MXNDArrayCreateEx 722 24.8500 0.0310 0.0640 0.0344
MXNDArrayWaitAll 642 13954.1699 5.6130 25.0610 21.7355
MXNDArraySyncCopyToCPU 1364 10729.6016 0.1180 24.1980 7.8663
MXNDArrayGetDType 1364 0.4980 0.0000 0.0010 0.0004
MXNDArrayGetShape 3450 1.6180 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1444 12.6520 0.0070 0.0280 0.0088
MXNDArrayFree 16206 165.4130 0.0000 0.9140 0.0102
MXNet C API Calls 73094 560.4510 487.3580 560.4510 36.5465
MXNet C API Concurrency 146187 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1284 0.6060 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6420 3.3520 0.0000 0.0230 0.0005
MXNDArraySetGradState 6420 3.2640 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1284 0.5390 0.0000 0.0010 0.0004
MXAutogradBackwardEx 642 354.7980 0.5250 0.9450 0.5526
MXNDArrayGetContext 10028 4.3710 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21182 544.0750 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2568 6294.4199 0.7010 3.7500 2.4511
FullyConnected 4332 419.2010 0.0080 0.3430 0.0968
argmax 1444 28.6620 0.0050 0.0480 0.0198
_backward_pick 1284 20.3220 0.0130 0.0320 0.0158
_backward_mul_scalar 1284 5.6130 0.0030 0.0220 0.0044
DeleteVariable 54240 382.4060 0.0010 0.9010 0.0071
Flatten 1444 59.8530 0.0080 0.1120 0.0414
Activation 5776 268.9150 0.0030 1.9440 0.0466
SetValueOp 1284 8.3000 0.0050 0.0180 0.0065
pick 1284 10.1090 0.0060 0.0280 0.0079
Pooling 2888 4059.0591 0.0640 3.1720 1.4055
Convolution 2888 16496.2324 0.2260 13.2050 5.7120
CopyCPU2CPU 1444 8.1630 0.0030 0.0360 0.0057
log_softmax 1284 39.9810 0.0150 0.0550 0.0311
_backward_copy 1284 66.2390 0.0220 0.0760 0.0516
_backward_Convolution 2568 23786.6406 2.6370 12.5610 9.2627
_mul_scalar 1284 4.6010 0.0020 0.0230 0.0036
_backward_log_softmax 1284 21.8510 0.0090 0.0400 0.0170
mean 4012 77.1520 0.0080 0.0730 0.0192
_backward_mean 1284 14.0950 0.0080 0.1030 0.0110
_backward_FullyConnected 3852 739.2690 0.0280 0.5940 0.1919
WaitForVar 2728 24.3750 0.0060 0.0280 0.0089
broadcast_equal 1444 7.2160 0.0030 0.0110 0.0050
sgd_update 12840 127.0850 0.0020 0.0790 0.0099
_backward_Activation 5136 380.0330 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 49982 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4338 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 642 2525.2871 0.3060 73.5850 3.9335
MXNDArrayCreateEx 723 24.8950 0.0310 0.0640 0.0344
MXNDArrayWaitAll 643 13975.9082 5.6130 25.0610 21.7355
MXNDArraySyncCopyToCPU 1366 10744.3291 0.1180 24.1980 7.8655
MXNDArrayGetDType 1366 0.5000 0.0000 0.0010 0.0004
MXNDArrayGetShape 3455 1.6220 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1446 12.6710 0.0070 0.0280 0.0088
MXNDArrayFree 16229 165.6560 0.0000 0.9140 0.0102
MXNet C API Calls 73201 560.5580 487.3580 560.5580 36.6000
MXNet C API Concurrency 146401 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1286 0.6080 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6430 3.3590 0.0000 0.0230 0.0005
MXNDArraySetGradState 6430 3.2730 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1286 0.5400 0.0000 0.0010 0.0004
MXAutogradBackwardEx 643 355.6850 0.5250 0.9450 0.5532
MXNDArrayGetContext 10042 4.3810 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21213 545.2350 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2572 6304.2031 0.7010 3.7500 2.4511
FullyConnected 4338 419.7910 0.0080 0.3430 0.0968
argmax 1446 28.7070 0.0050 0.0480 0.0199
_backward_pick 1286 20.3560 0.0130 0.0320 0.0158
_backward_mul_scalar 1286 5.6220 0.0030 0.0220 0.0044
DeleteVariable 54320 382.9630 0.0010 0.9010 0.0071
Flatten 1446 59.9320 0.0080 0.1120 0.0414
Activation 5784 269.2830 0.0030 1.9440 0.0466
SetValueOp 1286 8.3150 0.0050 0.0180 0.0065
pick 1286 10.1250 0.0060 0.0280 0.0079
Pooling 2892 4064.4890 0.0640 3.1720 1.4054
Convolution 2892 16522.1348 0.2260 13.2050 5.7130
CopyCPU2CPU 1446 8.1720 0.0030 0.0360 0.0057
log_softmax 1286 40.0440 0.0150 0.0550 0.0311
_backward_copy 1286 66.3380 0.0220 0.0760 0.0516
_backward_Convolution 2572 23823.4668 2.6370 12.5610 9.2626
_mul_scalar 1286 4.6080 0.0020 0.0230 0.0036
_backward_log_softmax 1286 21.8910 0.0090 0.0400 0.0170
mean 4018 77.2570 0.0080 0.0730 0.0192
_backward_mean 1286 14.1160 0.0080 0.1030 0.0110
_backward_FullyConnected 3858 740.7410 0.0280 0.5940 0.1920
WaitForVar 2732 24.4120 0.0060 0.0280 0.0089
broadcast_equal 1446 7.2270 0.0030 0.0110 0.0050
sgd_update 12860 127.3090 0.0020 0.0790 0.0099
_backward_Activation 5144 380.5960 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50056 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4344 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 643 2529.1909 0.3060 73.5850 3.9334
MXNDArrayCreateEx 724 24.9400 0.0310 0.0640 0.0344
MXNDArrayWaitAll 644 13997.6289 5.6130 25.0610 21.7354
MXNDArraySyncCopyToCPU 1368 10758.9766 0.1180 24.1980 7.8647
MXNDArrayGetDType 1368 0.5000 0.0000 0.0010 0.0004
MXNDArrayGetShape 3460 1.6260 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1448 12.6880 0.0070 0.0280 0.0088
MXNDArrayFree 16252 165.8970 0.0000 0.9140 0.0102
MXNet C API Calls 73308 560.6650 487.3580 560.6650 36.6535
MXNet C API Concurrency 146615 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1288 0.6090 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6440 3.3670 0.0000 0.0230 0.0005
MXNDArraySetGradState 6440 3.2810 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1288 0.5420 0.0000 0.0010 0.0004
MXAutogradBackwardEx 644 356.5700 0.5250 0.9450 0.5537
MXNDArrayGetContext 10056 4.3860 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21244 546.3950 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2576 6313.8452 0.7010 3.7500 2.4510
FullyConnected 4344 420.4210 0.0080 0.3430 0.0968
argmax 1448 28.7550 0.0050 0.0480 0.0199
_backward_pick 1288 20.3910 0.0130 0.0320 0.0158
_backward_mul_scalar 1288 5.6310 0.0030 0.0220 0.0044
DeleteVariable 54400 383.5110 0.0010 0.9010 0.0070
Flatten 1448 60.0120 0.0080 0.1120 0.0414
Activation 5792 269.6470 0.0030 1.9440 0.0466
SetValueOp 1288 8.3300 0.0050 0.0180 0.0065
pick 1288 10.1400 0.0060 0.0280 0.0079
Pooling 2896 4069.8940 0.0640 3.1720 1.4054
Convolution 2896 16548.0273 0.2260 13.2050 5.7141
CopyCPU2CPU 1448 8.1820 0.0030 0.0360 0.0057
log_softmax 1288 40.1090 0.0150 0.0550 0.0311
_backward_copy 1288 66.4390 0.0220 0.0760 0.0516
_backward_Convolution 2576 23860.3398 2.6370 12.5610 9.2626
_mul_scalar 1288 4.6150 0.0020 0.0230 0.0036
_backward_log_softmax 1288 21.9270 0.0090 0.0400 0.0170
mean 4024 77.3680 0.0080 0.0730 0.0192
_backward_mean 1288 14.1390 0.0080 0.1030 0.0110
_backward_FullyConnected 3864 742.1960 0.0280 0.5940 0.1921
WaitForVar 2736 24.4480 0.0060 0.0280 0.0089
broadcast_equal 1448 7.2360 0.0030 0.0110 0.0050
sgd_update 12880 127.5270 0.0020 0.0790 0.0099
_backward_Activation 5152 381.1580 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50130 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4350 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 644 2533.1221 0.3060 73.5850 3.9334
MXNDArrayCreateEx 725 24.9790 0.0310 0.0640 0.0345
MXNDArrayWaitAll 645 14019.4287 5.6130 25.0610 21.7355
MXNDArraySyncCopyToCPU 1370 10773.2842 0.1180 24.1980 7.8637
MXNDArrayGetDType 1370 0.5010 0.0000 0.0010 0.0004
MXNDArrayGetShape 3465 1.6290 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1450 12.7050 0.0070 0.0280 0.0088
MXNDArrayFree 16275 166.1290 0.0000 0.9140 0.0102
MXNet C API Calls 73415 560.7720 487.3580 560.7720 36.7070
MXNet C API Concurrency 146829 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1290 0.6100 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6450 3.3720 0.0000 0.0230 0.0005
MXNDArraySetGradState 6450 3.2870 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1290 0.5430 0.0000 0.0010 0.0004
MXAutogradBackwardEx 645 357.1110 0.5250 0.9450 0.5537
MXNDArrayGetContext 10070 4.3910 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21275 547.2130 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2580 6323.5190 0.7010 3.7500 2.4510
FullyConnected 4350 420.9500 0.0080 0.3430 0.0968
argmax 1450 28.7940 0.0050 0.0480 0.0199
_backward_pick 1290 20.4230 0.0130 0.0320 0.0158
_backward_mul_scalar 1290 5.6400 0.0030 0.0220 0.0044
DeleteVariable 54480 384.0520 0.0010 0.9010 0.0070
Flatten 1450 60.0920 0.0080 0.1120 0.0414
Activation 5800 269.9990 0.0030 1.9440 0.0466
SetValueOp 1290 8.3430 0.0050 0.0180 0.0065
pick 1290 10.1550 0.0060 0.0280 0.0079
Pooling 2900 4075.2710 0.0640 3.1720 1.4053
Convolution 2900 16570.3457 0.2260 13.2050 5.7139
CopyCPU2CPU 1450 8.1930 0.0030 0.0360 0.0057
log_softmax 1290 40.1680 0.0150 0.0550 0.0311
_backward_copy 1290 66.5420 0.0220 0.0760 0.0516
_backward_Convolution 2580 23897.3555 2.6370 12.5610 9.2625
_mul_scalar 1290 4.6220 0.0020 0.0230 0.0036
_backward_log_softmax 1290 21.9600 0.0090 0.0400 0.0170
mean 4030 77.4830 0.0080 0.0730 0.0192
_backward_mean 1290 14.1610 0.0080 0.1030 0.0110
_backward_FullyConnected 3870 743.2640 0.0280 0.5940 0.1921
WaitForVar 2740 24.4850 0.0060 0.0280 0.0089
broadcast_equal 1450 7.2460 0.0030 0.0110 0.0050
sgd_update 12900 127.7330 0.0020 0.0790 0.0099
_backward_Activation 5160 381.7210 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50204 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4356 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 645 2537.0681 0.3060 73.5850 3.9334
MXNDArrayCreateEx 726 25.0120 0.0310 0.0640 0.0345
MXNDArrayWaitAll 646 14041.2773 5.6130 25.0610 21.7357
MXNDArraySyncCopyToCPU 1372 10787.6748 0.1180 24.1980 7.8627
MXNDArrayGetDType 1372 0.5020 0.0000 0.0010 0.0004
MXNDArrayGetShape 3470 1.6320 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1452 12.7220 0.0070 0.0280 0.0088
MXNDArrayFree 16298 166.3500 0.0000 0.9140 0.0102
MXNet C API Calls 73522 560.8790 487.3580 560.8790 36.7605
MXNet C API Concurrency 147043 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1292 0.6110 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6460 3.3760 0.0000 0.0230 0.0005
MXNDArraySetGradState 6460 3.2920 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1292 0.5440 0.0000 0.0010 0.0004
MXAutogradBackwardEx 646 357.6450 0.5250 0.9450 0.5536
MXNDArrayGetContext 10084 4.3960 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21306 547.9810 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2584 6333.1748 0.7010 3.7500 2.4509
FullyConnected 4356 421.4780 0.0080 0.3430 0.0968
argmax 1452 28.8330 0.0050 0.0480 0.0199
_backward_pick 1292 20.4540 0.0130 0.0320 0.0158
_backward_mul_scalar 1292 5.6480 0.0030 0.0220 0.0044
DeleteVariable 54560 384.5890 0.0010 0.9010 0.0070
Flatten 1452 60.1720 0.0080 0.1120 0.0414
Activation 5808 270.3500 0.0030 1.9440 0.0465
SetValueOp 1292 8.3560 0.0050 0.0180 0.0065
pick 1292 10.1710 0.0060 0.0280 0.0079
Pooling 2904 4080.6350 0.0640 3.1720 1.4052
Convolution 2904 16592.5918 0.2260 13.2050 5.7137
CopyCPU2CPU 1452 8.2020 0.0030 0.0360 0.0056
log_softmax 1292 40.2270 0.0150 0.0550 0.0311
_backward_copy 1292 66.6470 0.0220 0.0760 0.0516
_backward_Convolution 2584 23934.4180 2.6370 12.5610 9.2625
_mul_scalar 1292 4.6290 0.0020 0.0230 0.0036
_backward_log_softmax 1292 21.9930 0.0090 0.0400 0.0170
mean 4036 77.5980 0.0080 0.0730 0.0192
_backward_mean 1292 14.1820 0.0080 0.1030 0.0110
_backward_FullyConnected 3876 744.3260 0.0280 0.5940 0.1920
WaitForVar 2744 24.5200 0.0060 0.0280 0.0089
broadcast_equal 1452 7.2560 0.0030 0.0110 0.0050
sgd_update 12920 127.9320 0.0020 0.0790 0.0099
_backward_Activation 5168 382.2790 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50278 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4362 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 646 2540.9651 0.3060 73.5850 3.9334
MXNDArrayCreateEx 727 25.0450 0.0310 0.0640 0.0344
MXNDArrayWaitAll 647 14062.9609 5.6130 25.0610 21.7356
MXNDArraySyncCopyToCPU 1374 10802.2383 0.1180 24.1980 7.8619
MXNDArrayGetDType 1374 0.5020 0.0000 0.0010 0.0004
MXNDArrayGetShape 3475 1.6350 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1454 12.7380 0.0070 0.0280 0.0088
MXNDArrayFree 16321 166.5740 0.0000 0.9140 0.0102
MXNet C API Calls 73629 560.9860 487.3580 560.9860 36.8140
MXNet C API Concurrency 147257 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1294 0.6110 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6470 3.3810 0.0000 0.0230 0.0005
MXNDArraySetGradState 6470 3.2970 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1294 0.5440 0.0000 0.0010 0.0004
MXAutogradBackwardEx 647 358.1790 0.5250 0.9450 0.5536
MXNDArrayGetContext 10098 4.4020 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21337 548.7780 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2588 6342.8779 0.7010 3.7500 2.4509
FullyConnected 4362 422.1010 0.0080 0.3430 0.0968
argmax 1454 28.8770 0.0050 0.0480 0.0199
_backward_pick 1294 20.4840 0.0130 0.0320 0.0158
_backward_mul_scalar 1294 5.6560 0.0030 0.0220 0.0044
DeleteVariable 54640 385.1210 0.0010 0.9010 0.0070
Flatten 1454 60.2500 0.0080 0.1120 0.0414
Activation 5816 270.7220 0.0030 1.9440 0.0465
SetValueOp 1294 8.3690 0.0050 0.0180 0.0065
pick 1294 10.1860 0.0060 0.0280 0.0079
Pooling 2908 4086.0449 0.0640 3.1720 1.4051
Convolution 2908 16614.7598 0.2260 13.2050 5.7135
CopyCPU2CPU 1454 8.2130 0.0030 0.0360 0.0056
log_softmax 1294 40.2860 0.0150 0.0550 0.0311
_backward_copy 1294 66.7490 0.0220 0.0760 0.0516
_backward_Convolution 2588 23971.2070 2.6370 12.5610 9.2624
_mul_scalar 1294 4.6370 0.0020 0.0230 0.0036
_backward_log_softmax 1294 22.0370 0.0090 0.0400 0.0170
mean 4042 77.7280 0.0080 0.0730 0.0192
_backward_mean 1294 14.2040 0.0080 0.1030 0.0110
_backward_FullyConnected 3882 745.4920 0.0280 0.5940 0.1920
WaitForVar 2748 24.5560 0.0060 0.0280 0.0089
broadcast_equal 1454 7.2650 0.0030 0.0110 0.0050
sgd_update 12940 128.1430 0.0020 0.0790 0.0099
_backward_Activation 5176 382.8470 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50352 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4368 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 647 2544.8601 0.3060 73.5850 3.9333
MXNDArrayCreateEx 728 25.0800 0.0310 0.0640 0.0345
MXNDArrayWaitAll 648 14084.6924 5.6130 25.0610 21.7356
MXNDArraySyncCopyToCPU 1376 10816.6426 0.1180 24.1980 7.8609
MXNDArrayGetDType 1376 0.5030 0.0000 0.0010 0.0004
MXNDArrayGetShape 3480 1.6370 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1456 12.7550 0.0070 0.0280 0.0088
MXNDArrayFree 16344 166.8010 0.0000 0.9140 0.0102
MXNet C API Calls 73736 561.0930 487.3580 561.0930 36.8675
MXNet C API Concurrency 147471 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1296 0.6130 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6480 3.3870 0.0000 0.0230 0.0005
MXNDArraySetGradState 6480 3.3020 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1296 0.5450 0.0000 0.0010 0.0004
MXAutogradBackwardEx 648 358.7100 0.5250 0.9450 0.5536
MXNDArrayGetContext 10112 4.4050 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21368 549.5500 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2592 6352.5942 0.7010 3.7500 2.4508
FullyConnected 4368 422.7020 0.0080 0.3430 0.0968
argmax 1456 28.9160 0.0050 0.0480 0.0199
_backward_pick 1296 20.5160 0.0130 0.0320 0.0158
_backward_mul_scalar 1296 5.6650 0.0030 0.0220 0.0044
DeleteVariable 54720 385.6370 0.0010 0.9010 0.0070
Flatten 1456 60.3280 0.0080 0.1120 0.0414
Activation 5824 271.0920 0.0030 1.9440 0.0465
SetValueOp 1296 8.3800 0.0050 0.0180 0.0065
pick 1296 10.2010 0.0060 0.0280 0.0079
Pooling 2912 4091.4060 0.0640 3.1720 1.4050
Convolution 2912 16636.8242 0.2260 13.2050 5.7132
CopyCPU2CPU 1456 8.2230 0.0030 0.0360 0.0056
log_softmax 1296 40.3570 0.0150 0.0550 0.0311
_backward_copy 1296 66.8500 0.0220 0.0760 0.0516
_backward_Convolution 2592 24007.9766 2.6370 12.5610 9.2623
_mul_scalar 1296 4.6440 0.0020 0.0230 0.0036
_backward_log_softmax 1296 22.0690 0.0090 0.0400 0.0170
mean 4048 77.8320 0.0080 0.0730 0.0192
_backward_mean 1296 14.2260 0.0080 0.1030 0.0110
_backward_FullyConnected 3888 746.6830 0.0280 0.5940 0.1920
WaitForVar 2752 24.5910 0.0060 0.0280 0.0089
broadcast_equal 1456 7.2740 0.0030 0.0110 0.0050
sgd_update 12960 128.2990 0.0020 0.0790 0.0099
_backward_Activation 5184 383.4620 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50426 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4374 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 648 2548.7629 0.3060 73.5850 3.9333
MXNDArrayCreateEx 729 25.1140 0.0310 0.0640 0.0344
MXNDArrayWaitAll 649 14106.5527 5.6130 25.0610 21.7358
MXNDArraySyncCopyToCPU 1378 10831.0146 0.1180 24.1980 7.8600
MXNDArrayGetDType 1378 0.5040 0.0000 0.0010 0.0004
MXNDArrayGetShape 3485 1.6410 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1458 12.7720 0.0070 0.0280 0.0088
MXNDArrayFree 16367 167.0320 0.0000 0.9140 0.0102
MXNet C API Calls 73843 561.2000 487.3580 561.2000 36.9210
MXNet C API Concurrency 147685 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1298 0.6130 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6490 3.3910 0.0000 0.0230 0.0005
MXNDArraySetGradState 6490 3.3060 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1298 0.5460 0.0000 0.0010 0.0004
MXAutogradBackwardEx 649 359.2710 0.5250 0.9450 0.5536
MXNDArrayGetContext 10126 4.4130 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21399 550.3220 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2596 6362.2339 0.7010 3.7500 2.4508
FullyConnected 4374 423.2350 0.0080 0.3430 0.0968
argmax 1458 28.9540 0.0050 0.0480 0.0199
_backward_pick 1298 20.5470 0.0130 0.0320 0.0158
_backward_mul_scalar 1298 5.6740 0.0030 0.0220 0.0044
DeleteVariable 54800 386.1490 0.0010 0.9010 0.0070
Flatten 1458 60.4110 0.0080 0.1120 0.0414
Activation 5832 271.4430 0.0030 1.9440 0.0465
SetValueOp 1298 8.3920 0.0050 0.0180 0.0065
pick 1298 10.2170 0.0060 0.0280 0.0079
Pooling 2916 4096.8188 0.0640 3.1720 1.4049
Convolution 2916 16658.9902 0.2260 13.2050 5.7130
CopyCPU2CPU 1458 8.2320 0.0030 0.0360 0.0056
log_softmax 1298 40.4160 0.0150 0.0550 0.0311
_backward_copy 1298 66.9520 0.0220 0.0760 0.0516
_backward_Convolution 2596 24045.0684 2.6370 12.5610 9.2624
_mul_scalar 1298 4.6510 0.0020 0.0230 0.0036
_backward_log_softmax 1298 22.1010 0.0090 0.0400 0.0170
mean 4054 77.9470 0.0080 0.0730 0.0192
_backward_mean 1298 14.2470 0.0080 0.1030 0.0110
_backward_FullyConnected 3894 747.7490 0.0280 0.5940 0.1920
WaitForVar 2756 24.6270 0.0060 0.0280 0.0089
broadcast_equal 1458 7.2830 0.0030 0.0110 0.0050
sgd_update 12980 128.5010 0.0020 0.0790 0.0099
_backward_Activation 5192 384.0270 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50500 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4380 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 649 2552.6790 0.3060 73.5850 3.9332
MXNDArrayCreateEx 730 25.1480 0.0310 0.0640 0.0344
MXNDArrayWaitAll 650 14128.3623 5.6130 25.0610 21.7359
MXNDArraySyncCopyToCPU 1380 10845.3701 0.1180 24.1980 7.8590
MXNDArrayGetDType 1380 0.5050 0.0000 0.0010 0.0004
MXNDArrayGetShape 3490 1.6430 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1460 12.7900 0.0070 0.0280 0.0088
MXNDArrayFree 16390 167.2790 0.0000 0.9140 0.0102
MXNet C API Calls 73950 561.3070 487.3580 561.3070 36.9745
MXNet C API Concurrency 147899 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1300 0.6140 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6500 3.3960 0.0000 0.0230 0.0005
MXNDArraySetGradState 6500 3.3080 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1300 0.5480 0.0000 0.0010 0.0004
MXAutogradBackwardEx 650 359.8080 0.5250 0.9450 0.5536
MXNDArrayGetContext 10140 4.4180 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21430 551.1050 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2600 6371.9312 0.7010 3.7500 2.4507
FullyConnected 4380 423.7740 0.0080 0.3430 0.0968
argmax 1460 28.9940 0.0050 0.0480 0.0199
_backward_pick 1300 20.5770 0.0130 0.0320 0.0158
_backward_mul_scalar 1300 5.6820 0.0030 0.0220 0.0044
DeleteVariable 54880 386.7130 0.0010 0.9010 0.0070
Flatten 1460 60.4930 0.0080 0.1120 0.0414
Activation 5840 271.7970 0.0030 1.9440 0.0465
SetValueOp 1300 8.4040 0.0050 0.0180 0.0065
pick 1300 10.2330 0.0060 0.0280 0.0079
Pooling 2920 4102.1938 0.0640 3.1720 1.4049
Convolution 2920 16681.2168 0.2260 13.2050 5.7127
CopyCPU2CPU 1460 8.2420 0.0030 0.0360 0.0056
log_softmax 1300 40.4750 0.0150 0.0550 0.0311
_backward_copy 1300 67.0570 0.0220 0.0760 0.0516
_backward_Convolution 2600 24082.1055 2.6370 12.5610 9.2623
_mul_scalar 1300 4.6580 0.0020 0.0230 0.0036
_backward_log_softmax 1300 22.1340 0.0090 0.0400 0.0170
mean 4060 78.0630 0.0080 0.0730 0.0192
_backward_mean 1300 14.2680 0.0080 0.1030 0.0110
_backward_FullyConnected 3900 748.8230 0.0280 0.5940 0.1920
WaitForVar 2760 24.6610 0.0060 0.0280 0.0089
broadcast_equal 1460 7.2930 0.0030 0.0110 0.0050
sgd_update 13000 128.7030 0.0020 0.0790 0.0099
_backward_Activation 5200 384.5950 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50574 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4386 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 650 2556.6250 0.3060 73.5850 3.9333
MXNDArrayCreateEx 731 25.1820 0.0310 0.0640 0.0344
MXNDArrayWaitAll 651 14150.0811 5.6130 25.0610 21.7359
MXNDArraySyncCopyToCPU 1382 10859.8604 0.1180 24.1980 7.8581
MXNDArrayGetDType 1382 0.5050 0.0000 0.0010 0.0004
MXNDArrayGetShape 3495 1.6450 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1462 12.8080 0.0070 0.0280 0.0088
MXNDArrayFree 16413 167.5110 0.0000 0.9140 0.0102
MXNet C API Calls 74057 561.4140 487.3580 561.4140 37.0280
MXNet C API Concurrency 148113 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1302 0.6140 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6510 3.4020 0.0000 0.0230 0.0005
MXNDArraySetGradState 6510 3.3160 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1302 0.5480 0.0000 0.0010 0.0004
MXAutogradBackwardEx 651 360.3470 0.5250 0.9450 0.5535
MXNDArrayGetContext 10154 4.4250 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21461 551.8750 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2604 6381.6260 0.7010 3.7500 2.4507
FullyConnected 4386 424.3680 0.0080 0.3430 0.0968
argmax 1462 29.0370 0.0050 0.0480 0.0199
_backward_pick 1302 20.6120 0.0130 0.0320 0.0158
_backward_mul_scalar 1302 5.6910 0.0030 0.0220 0.0044
DeleteVariable 54960 387.2500 0.0010 0.9010 0.0070
Flatten 1462 60.5700 0.0080 0.1120 0.0414
Activation 5848 272.1690 0.0030 1.9440 0.0465
SetValueOp 1302 8.4170 0.0050 0.0180 0.0065
pick 1302 10.2500 0.0060 0.0280 0.0079
Pooling 2924 4107.5811 0.0640 3.1720 1.4048
Convolution 2924 16703.3984 0.2260 13.2050 5.7125
CopyCPU2CPU 1462 8.2510 0.0030 0.0360 0.0056
log_softmax 1302 40.5390 0.0150 0.0550 0.0311
_backward_copy 1302 67.1580 0.0220 0.0760 0.0516
_backward_Convolution 2604 24118.9082 2.6370 12.5610 9.2623
_mul_scalar 1302 4.6660 0.0020 0.0230 0.0036
_backward_log_softmax 1302 22.1720 0.0090 0.0400 0.0170
mean 4066 78.1680 0.0080 0.0730 0.0192
_backward_mean 1302 14.2890 0.0080 0.1030 0.0110
_backward_FullyConnected 3906 750.0040 0.0280 0.5940 0.1920
WaitForVar 2764 24.6990 0.0060 0.0280 0.0089
broadcast_equal 1462 7.3020 0.0030 0.0110 0.0050
sgd_update 13020 128.9210 0.0020 0.0790 0.0099
_backward_Activation 5208 385.1460 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50648 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4392 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 651 2560.5430 0.3060 73.5850 3.9332
MXNDArrayCreateEx 732 25.2160 0.0310 0.0640 0.0344
MXNDArrayWaitAll 652 14171.7920 5.6130 25.0610 21.7359
MXNDArraySyncCopyToCPU 1384 10874.3428 0.1180 24.1980 7.8572
MXNDArrayGetDType 1384 0.5050 0.0000 0.0010 0.0004
MXNDArrayGetShape 3500 1.6480 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1464 12.8260 0.0070 0.0280 0.0088
MXNDArrayFree 16436 167.7450 0.0000 0.9140 0.0102
MXNet C API Calls 74164 561.5210 487.3580 561.5210 37.0815
MXNet C API Concurrency 148327 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1304 0.6160 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6520 3.4070 0.0000 0.0230 0.0005
MXNDArraySetGradState 6520 3.3200 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1304 0.5500 0.0000 0.0010 0.0004
MXAutogradBackwardEx 652 360.8830 0.5250 0.9450 0.5535
MXNDArrayGetContext 10168 4.4340 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21492 552.6740 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2608 6391.3921 0.7010 3.7500 2.4507
FullyConnected 4392 424.9820 0.0080 0.3430 0.0968
argmax 1464 29.0800 0.0050 0.0480 0.0199
_backward_pick 1304 20.6410 0.0130 0.0320 0.0158
_backward_mul_scalar 1304 5.6990 0.0030 0.0220 0.0044
DeleteVariable 55040 387.7980 0.0010 0.9010 0.0070
Flatten 1464 60.6540 0.0080 0.1120 0.0414
Activation 5856 272.5460 0.0030 1.9440 0.0465
SetValueOp 1304 8.4290 0.0050 0.0180 0.0065
pick 1304 10.2650 0.0060 0.0280 0.0079
Pooling 2928 4112.9551 0.0640 3.1720 1.4047
Convolution 2928 16725.4746 0.2260 13.2050 5.7123
CopyCPU2CPU 1464 8.2610 0.0030 0.0360 0.0056
log_softmax 1304 40.5980 0.0150 0.0550 0.0311
_backward_copy 1304 67.2590 0.0220 0.0760 0.0516
_backward_Convolution 2608 24155.6875 2.6370 12.5610 9.2621
_mul_scalar 1304 4.6730 0.0020 0.0230 0.0036
_backward_log_softmax 1304 22.2140 0.0090 0.0400 0.0170
mean 4072 78.2710 0.0080 0.0730 0.0192
_backward_mean 1304 14.3100 0.0080 0.1030 0.0110
_backward_FullyConnected 3912 751.1840 0.0280 0.5940 0.1920
WaitForVar 2768 24.7340 0.0060 0.0280 0.0089
broadcast_equal 1464 7.3110 0.0030 0.0110 0.0050
sgd_update 13040 129.1450 0.0020 0.0790 0.0099
_backward_Activation 5216 385.7130 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50722 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4398 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 652 2564.4519 0.3060 73.5850 3.9332
MXNDArrayCreateEx 733 25.2500 0.0310 0.0640 0.0344
MXNDArrayWaitAll 653 14193.6680 5.6130 25.0610 21.7361
MXNDArraySyncCopyToCPU 1386 10888.6846 0.1180 24.1980 7.8562
MXNDArrayGetDType 1386 0.5050 0.0000 0.0010 0.0004
MXNDArrayGetShape 3505 1.6490 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1466 12.8430 0.0070 0.0280 0.0088
MXNDArrayFree 16459 167.9770 0.0000 0.9140 0.0102
MXNet C API Calls 74271 561.6280 487.3580 561.6280 37.1350
MXNet C API Concurrency 148541 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1306 0.6180 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6530 3.4110 0.0000 0.0230 0.0005
MXNDArraySetGradState 6530 3.3250 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1306 0.5520 0.0000 0.0010 0.0004
MXAutogradBackwardEx 653 361.4220 0.5250 0.9450 0.5535
MXNDArrayGetContext 10182 4.4400 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21523 553.4590 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2612 6401.1030 0.7010 3.7500 2.4507
FullyConnected 4398 425.5120 0.0080 0.3430 0.0968
argmax 1466 29.1180 0.0050 0.0480 0.0199
_backward_pick 1306 20.6730 0.0130 0.0320 0.0158
_backward_mul_scalar 1306 5.7070 0.0030 0.0220 0.0044
DeleteVariable 55120 388.3300 0.0010 0.9010 0.0070
Flatten 1466 60.7360 0.0080 0.1120 0.0414
Activation 5864 272.9000 0.0030 1.9440 0.0465
SetValueOp 1306 8.4420 0.0050 0.0180 0.0065
pick 1306 10.2800 0.0060 0.0280 0.0079
Pooling 2932 4118.3159 0.0640 3.1720 1.4046
Convolution 2932 16747.6816 0.2260 13.2050 5.7120
CopyCPU2CPU 1466 8.2710 0.0030 0.0360 0.0056
log_softmax 1306 40.6570 0.0150 0.0550 0.0311
_backward_copy 1306 67.3620 0.0220 0.0760 0.0516
_backward_Convolution 2612 24192.7559 2.6370 12.5610 9.2622
_mul_scalar 1306 4.6800 0.0020 0.0230 0.0036
_backward_log_softmax 1306 22.2460 0.0090 0.0400 0.0170
mean 4078 78.3870 0.0080 0.0730 0.0192
_backward_mean 1306 14.3310 0.0080 0.1030 0.0110
_backward_FullyConnected 3918 752.3010 0.0280 0.5940 0.1920
WaitForVar 2772 24.7680 0.0060 0.0280 0.0089
broadcast_equal 1466 7.3200 0.0030 0.0110 0.0050
sgd_update 13060 129.2950 0.0020 0.0790 0.0099
_backward_Activation 5224 386.3300 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50796 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4404 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 653 2568.3569 0.3060 73.5850 3.9332
MXNDArrayCreateEx 734 25.2840 0.0310 0.0640 0.0344
MXNDArrayWaitAll 654 14215.4814 5.6130 25.0610 21.7362
MXNDArraySyncCopyToCPU 1388 10903.1152 0.1180 24.1980 7.8553
MXNDArrayGetDType 1388 0.5050 0.0000 0.0010 0.0004
MXNDArrayGetShape 3510 1.6520 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1468 12.8590 0.0070 0.0280 0.0088
MXNDArrayFree 16482 168.2040 0.0000 0.9140 0.0102
MXNet C API Calls 74378 561.7350 487.3580 561.7350 37.1885
MXNet C API Concurrency 148755 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1308 0.6190 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6540 3.4160 0.0000 0.0230 0.0005
MXNDArraySetGradState 6540 3.3290 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1308 0.5520 0.0000 0.0010 0.0004
MXAutogradBackwardEx 654 361.9580 0.5250 0.9450 0.5535
MXNDArrayGetContext 10196 4.4450 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21554 554.2320 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2616 6410.7749 0.7010 3.7500 2.4506
FullyConnected 4404 426.0350 0.0080 0.3430 0.0967
argmax 1468 29.1550 0.0050 0.0480 0.0199
_backward_pick 1308 20.7040 0.0130 0.0320 0.0158
_backward_mul_scalar 1308 5.7160 0.0030 0.0220 0.0044
DeleteVariable 55200 388.8510 0.0010 0.9010 0.0070
Flatten 1468 60.8160 0.0080 0.1120 0.0414
Activation 5872 273.2510 0.0030 1.9440 0.0465
SetValueOp 1308 8.4540 0.0050 0.0180 0.0065
pick 1308 10.2960 0.0060 0.0280 0.0079
Pooling 2936 4123.6982 0.0640 3.1720 1.4045
Convolution 2936 16769.9004 0.2260 13.2050 5.7118
CopyCPU2CPU 1468 8.2820 0.0030 0.0360 0.0056
log_softmax 1308 40.7160 0.0150 0.0550 0.0311
_backward_copy 1308 67.4660 0.0220 0.0760 0.0516
_backward_Convolution 2616 24229.7949 2.6370 12.5610 9.2622
_mul_scalar 1308 4.6870 0.0020 0.0230 0.0036
_backward_log_softmax 1308 22.2780 0.0090 0.0400 0.0170
mean 4084 78.4990 0.0080 0.0730 0.0192
_backward_mean 1308 14.3520 0.0080 0.1030 0.0110
_backward_FullyConnected 3924 753.3760 0.0280 0.5940 0.1920
WaitForVar 2776 24.8020 0.0060 0.0280 0.0089
broadcast_equal 1468 7.3300 0.0030 0.0110 0.0050
sgd_update 13080 129.4980 0.0020 0.0790 0.0099
_backward_Activation 5232 386.9050 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50870 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4410 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 654 2572.2671 0.3060 73.5850 3.9331
MXNDArrayCreateEx 735 25.3180 0.0310 0.0640 0.0344
MXNDArrayWaitAll 655 14237.1445 5.6130 25.0610 21.7361
MXNDArraySyncCopyToCPU 1390 10917.6846 0.1180 24.1980 7.8544
MXNDArrayGetDType 1390 0.5070 0.0000 0.0010 0.0004
MXNDArrayGetShape 3515 1.6530 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1470 12.8760 0.0070 0.0280 0.0088
MXNDArrayFree 16505 168.4300 0.0000 0.9140 0.0102
MXNet C API Calls 74485 561.8420 487.3580 561.8420 37.2420
MXNet C API Concurrency 148969 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1310 0.6190 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6550 3.4180 0.0000 0.0230 0.0005
MXNDArraySetGradState 6550 3.3360 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1310 0.5530 0.0000 0.0010 0.0004
MXAutogradBackwardEx 655 362.4960 0.5250 0.9450 0.5534
MXNDArrayGetContext 10210 4.4510 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21585 554.9970 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2620 6420.4458 0.7010 3.7500 2.4506
FullyConnected 4410 426.6490 0.0080 0.3430 0.0967
argmax 1470 29.1980 0.0050 0.0480 0.0199
_backward_pick 1310 20.7350 0.0130 0.0320 0.0158
_backward_mul_scalar 1310 5.7250 0.0030 0.0220 0.0044
DeleteVariable 55280 389.4090 0.0010 0.9010 0.0070
Flatten 1470 60.8970 0.0080 0.1120 0.0414
Activation 5880 273.6210 0.0030 1.9440 0.0465
SetValueOp 1310 8.4650 0.0050 0.0180 0.0065
pick 1310 10.3120 0.0060 0.0280 0.0079
Pooling 2940 4129.0732 0.0640 3.1720 1.4044
Convolution 2940 16792.0645 0.2260 13.2050 5.7116
CopyCPU2CPU 1470 8.2910 0.0030 0.0360 0.0056
log_softmax 1310 40.7760 0.0150 0.0550 0.0311
_backward_copy 1310 67.5700 0.0220 0.0760 0.0516
_backward_Convolution 2620 24266.5117 2.6370 12.5610 9.2620
_mul_scalar 1310 4.6940 0.0020 0.0230 0.0036
_backward_log_softmax 1310 22.3190 0.0090 0.0400 0.0170
mean 4090 78.6280 0.0080 0.0730 0.0192
_backward_mean 1310 14.3730 0.0080 0.1030 0.0110
_backward_FullyConnected 3930 754.5500 0.0280 0.5940 0.1920
WaitForVar 2780 24.8370 0.0060 0.0280 0.0089
broadcast_equal 1470 7.3410 0.0030 0.0110 0.0050
sgd_update 13100 129.6640 0.0020 0.0790 0.0099
_backward_Activation 5240 387.5220 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 50944 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4416 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 655 2576.1760 0.3060 73.5850 3.9331
MXNDArrayCreateEx 736 25.3520 0.0310 0.0640 0.0344
MXNDArrayWaitAll 656 14258.8076 5.6130 25.0610 21.7360
MXNDArraySyncCopyToCPU 1392 10932.1279 0.1180 24.1980 7.8535
MXNDArrayGetDType 1392 0.5080 0.0000 0.0010 0.0004
MXNDArrayGetShape 3520 1.6540 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1472 12.8940 0.0070 0.0280 0.0088
MXNDArrayFree 16528 168.6730 0.0000 0.9140 0.0102
MXNet C API Calls 74592 561.9490 487.3580 561.9490 37.2955
MXNet C API Concurrency 149183 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1312 0.6200 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6560 3.4210 0.0000 0.0230 0.0005
MXNDArraySetGradState 6560 3.3410 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1312 0.5550 0.0000 0.0010 0.0004
MXAutogradBackwardEx 656 363.0490 0.5250 0.9450 0.5534
MXNDArrayGetContext 10224 4.4590 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21616 555.7680 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2624 6430.0630 0.7010 3.7500 2.4505
FullyConnected 4416 427.2380 0.0080 0.3430 0.0967
argmax 1472 29.2350 0.0050 0.0480 0.0199
_backward_pick 1312 20.7680 0.0130 0.0320 0.0158
_backward_mul_scalar 1312 5.7330 0.0030 0.0220 0.0044
DeleteVariable 55360 389.9220 0.0010 0.9010 0.0070
Flatten 1472 60.9770 0.0080 0.1120 0.0414
Activation 5888 273.9930 0.0030 1.9440 0.0465
SetValueOp 1312 8.4780 0.0050 0.0180 0.0065
pick 1312 10.3290 0.0060 0.0280 0.0079
Pooling 2944 4134.4302 0.0640 3.1720 1.4044
Convolution 2944 16814.1406 0.2260 13.2050 5.7113
CopyCPU2CPU 1472 8.3010 0.0030 0.0360 0.0056
log_softmax 1312 40.8480 0.0150 0.0550 0.0311
_backward_copy 1312 67.6720 0.0220 0.0760 0.0516
_backward_Convolution 2624 24303.3125 2.6370 12.5610 9.2619
_mul_scalar 1312 4.7010 0.0020 0.0230 0.0036
_backward_log_softmax 1312 22.3510 0.0090 0.0400 0.0170
mean 4096 78.7390 0.0080 0.0730 0.0192
_backward_mean 1312 14.3940 0.0080 0.1030 0.0110
_backward_FullyConnected 3936 755.7360 0.0280 0.5940 0.1920
WaitForVar 2784 24.8730 0.0060 0.0280 0.0089
broadcast_equal 1472 7.3510 0.0030 0.0110 0.0050
sgd_update 13120 129.8890 0.0020 0.0790 0.0099
_backward_Activation 5248 388.0950 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51018 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4422 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 656 2576.7180 0.3060 73.5850 3.9279
MXNDArrayCreateEx 737 25.3860 0.0310 0.0640 0.0344
MXNDArrayWaitAll 657 14280.7217 5.6130 25.0610 21.7363
MXNDArraySyncCopyToCPU 1394 10946.3555 0.1180 24.1980 7.8525
MXNDArrayGetDType 1394 0.5090 0.0000 0.0010 0.0004
MXNDArrayGetShape 3525 1.6560 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1474 12.9120 0.0070 0.0280 0.0088
MXNDArrayFree 16551 168.9120 0.0000 0.9140 0.0102
MXNet C API Calls 74699 562.0560 487.3580 562.0560 37.3490
MXNet C API Concurrency 149397 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1314 0.6200 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6570 3.4260 0.0000 0.0230 0.0005
MXNDArraySetGradState 6570 3.3450 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1314 0.5560 0.0000 0.0010 0.0004
MXAutogradBackwardEx 657 363.6120 0.5250 0.9450 0.5534
MXNDArrayGetContext 10238 4.4650 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21647 556.5760 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2628 6439.8091 0.7010 3.7500 2.4505
FullyConnected 4422 427.7740 0.0080 0.3430 0.0967
argmax 1474 29.2750 0.0050 0.0480 0.0199
_backward_pick 1314 20.8000 0.0130 0.0320 0.0158
_backward_mul_scalar 1314 5.7410 0.0030 0.0220 0.0044
DeleteVariable 55440 390.4920 0.0010 0.9010 0.0070
Flatten 1474 61.0620 0.0080 0.1120 0.0414
Activation 5896 274.3490 0.0030 1.9440 0.0465
SetValueOp 1314 8.4910 0.0050 0.0180 0.0065
pick 1314 10.3440 0.0060 0.0280 0.0079
Pooling 2948 4139.7549 0.0640 3.1720 1.4043
Convolution 2948 16836.3281 0.2260 13.2050 5.7111
CopyCPU2CPU 1474 8.3110 0.0030 0.0360 0.0056
log_softmax 1314 40.9080 0.0150 0.0550 0.0311
_backward_copy 1314 67.7720 0.0220 0.0760 0.0516
_backward_Convolution 2628 24340.4434 2.6370 12.5610 9.2620
_mul_scalar 1314 4.7090 0.0020 0.0230 0.0036
_backward_log_softmax 1314 22.3820 0.0090 0.0400 0.0170
mean 4102 78.8600 0.0080 0.0730 0.0192
_backward_mean 1314 14.4160 0.0080 0.1030 0.0110
_backward_FullyConnected 3942 756.8050 0.0280 0.5940 0.1920
WaitForVar 2788 24.9090 0.0060 0.0280 0.0089
broadcast_equal 1474 7.3610 0.0030 0.0110 0.0050
sgd_update 13140 130.0760 0.0020 0.0790 0.0099
_backward_Activation 5256 388.7290 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51092 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4428 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 657 2580.6201 0.3060 73.5850 3.9279
MXNDArrayCreateEx 738 25.4200 0.0310 0.0640 0.0344
MXNDArrayWaitAll 658 14302.5889 5.6130 25.0610 21.7365
MXNDArraySyncCopyToCPU 1396 10960.7578 0.1180 24.1980 7.8515
MXNDArrayGetDType 1396 0.5100 0.0000 0.0010 0.0004
MXNDArrayGetShape 3530 1.6580 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1476 12.9290 0.0070 0.0280 0.0088
MXNDArrayFree 16574 169.1430 0.0000 0.9140 0.0102
MXNet C API Calls 74806 562.1630 487.3580 562.1630 37.4025
MXNet C API Concurrency 149611 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1316 0.6210 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6580 3.4310 0.0000 0.0230 0.0005
MXNDArraySetGradState 6580 3.3490 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1316 0.5570 0.0000 0.0010 0.0004
MXAutogradBackwardEx 658 364.1720 0.5250 0.9450 0.5535
MXNDArrayGetContext 10252 4.4720 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21678 557.3510 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2632 6449.4971 0.7010 3.7500 2.4504
FullyConnected 4428 428.3010 0.0080 0.3430 0.0967
argmax 1476 29.3150 0.0050 0.0480 0.0199
_backward_pick 1316 20.8320 0.0130 0.0320 0.0158
_backward_mul_scalar 1316 5.7500 0.0030 0.0220 0.0044
DeleteVariable 55520 391.0280 0.0010 0.9010 0.0070
Flatten 1476 61.1440 0.0080 0.1120 0.0414
Activation 5904 274.7050 0.0030 1.9440 0.0465
SetValueOp 1316 8.5040 0.0050 0.0180 0.0065
pick 1316 10.3590 0.0060 0.0280 0.0079
Pooling 2952 4145.1929 0.0640 3.1720 1.4042
Convolution 2952 16858.5586 0.2260 13.2050 5.7109
CopyCPU2CPU 1476 8.3210 0.0030 0.0360 0.0056
log_softmax 1316 40.9680 0.0150 0.0550 0.0311
_backward_copy 1316 67.8760 0.0220 0.0760 0.0516
_backward_Convolution 2632 24377.4863 2.6370 12.5610 9.2620
_mul_scalar 1316 4.7160 0.0020 0.0230 0.0036
_backward_log_softmax 1316 22.4140 0.0090 0.0400 0.0170
mean 4108 78.9760 0.0080 0.0730 0.0192
_backward_mean 1316 14.4370 0.0080 0.1030 0.0110
_backward_FullyConnected 3948 757.8900 0.0280 0.5940 0.1920
WaitForVar 2792 24.9440 0.0060 0.0280 0.0089
broadcast_equal 1476 7.3710 0.0030 0.0110 0.0050
sgd_update 13160 130.2250 0.0020 0.0790 0.0099
_backward_Activation 5264 389.3260 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51166 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4434 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 658 2584.5330 0.3060 73.5850 3.9279
MXNDArrayCreateEx 739 25.4540 0.0310 0.0640 0.0344
MXNDArrayWaitAll 659 14324.3281 5.6130 25.0610 21.7365
MXNDArraySyncCopyToCPU 1398 10975.2744 0.1180 24.1980 7.8507
MXNDArrayGetDType 1398 0.5120 0.0000 0.0010 0.0004
MXNDArrayGetShape 3535 1.6610 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1478 12.9470 0.0070 0.0280 0.0088
MXNDArrayFree 16597 169.3740 0.0000 0.9140 0.0102
MXNet C API Calls 74913 562.2700 487.3580 562.2700 37.4560
MXNet C API Concurrency 149825 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1318 0.6230 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6590 3.4370 0.0000 0.0230 0.0005
MXNDArraySetGradState 6590 3.3550 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1318 0.5590 0.0000 0.0010 0.0004
MXAutogradBackwardEx 659 364.7100 0.5250 0.9450 0.5534
MXNDArrayGetContext 10266 4.4800 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21709 558.1240 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2636 6459.2310 0.7010 3.7500 2.4504
FullyConnected 4434 428.9250 0.0080 0.3430 0.0967
argmax 1478 29.3600 0.0050 0.0480 0.0199
_backward_pick 1318 20.8640 0.0130 0.0320 0.0158
_backward_mul_scalar 1318 5.7590 0.0030 0.0220 0.0044
DeleteVariable 55600 391.5550 0.0010 0.9010 0.0070
Flatten 1478 61.2270 0.0080 0.1120 0.0414
Activation 5912 275.0660 0.0030 1.9440 0.0465
SetValueOp 1318 8.5170 0.0050 0.0180 0.0065
pick 1318 10.3760 0.0060 0.0280 0.0079
Pooling 2956 4150.5850 0.0640 3.1720 1.4041
Convolution 2956 16880.7520 0.2260 13.2050 5.7107
CopyCPU2CPU 1478 8.3320 0.0030 0.0360 0.0056
log_softmax 1318 41.0290 0.0150 0.0550 0.0311
_backward_copy 1318 67.9780 0.0220 0.0760 0.0516
_backward_Convolution 2636 24414.2480 2.6370 12.5610 9.2619
_mul_scalar 1318 4.7230 0.0020 0.0230 0.0036
_backward_log_softmax 1318 22.4550 0.0090 0.0400 0.0170
mean 4114 79.0870 0.0080 0.0730 0.0192
_backward_mean 1318 14.4590 0.0080 0.1030 0.0110
_backward_FullyConnected 3954 759.0690 0.0280 0.5940 0.1920
WaitForVar 2796 24.9790 0.0060 0.0280 0.0089
broadcast_equal 1478 7.3810 0.0030 0.0110 0.0050
sgd_update 13180 130.4240 0.0020 0.0790 0.0099
_backward_Activation 5272 389.9020 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51240 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4440 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 659 2588.4541 0.3060 73.5850 3.9279
MXNDArrayCreateEx 740 25.4880 0.0310 0.0640 0.0344
MXNDArrayWaitAll 660 14346.0693 5.6130 25.0610 21.7365
MXNDArraySyncCopyToCPU 1400 10989.7393 0.1180 24.1980 7.8498
MXNDArrayGetDType 1400 0.5140 0.0000 0.0010 0.0004
MXNDArrayGetShape 3540 1.6610 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1480 12.9650 0.0070 0.0280 0.0088
MXNDArrayFree 16620 169.6150 0.0000 0.9140 0.0102
MXNet C API Calls 75020 562.3770 487.3580 562.3770 37.5095
MXNet C API Concurrency 150039 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1320 0.6240 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6600 3.4430 0.0000 0.0230 0.0005
MXNDArraySetGradState 6600 3.3610 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1320 0.5600 0.0000 0.0010 0.0004
MXAutogradBackwardEx 660 365.2510 0.5250 0.9450 0.5534
MXNDArrayGetContext 10280 4.4870 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21740 558.8930 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2640 6468.9839 0.7010 3.7500 2.4504
FullyConnected 4440 429.5380 0.0080 0.3430 0.0967
argmax 1480 29.3970 0.0050 0.0480 0.0199
_backward_pick 1320 20.8940 0.0130 0.0320 0.0158
_backward_mul_scalar 1320 5.7690 0.0030 0.0220 0.0044
DeleteVariable 55680 392.1090 0.0010 0.9010 0.0070
Flatten 1480 61.3070 0.0080 0.1120 0.0414
Activation 5920 275.4430 0.0030 1.9440 0.0465
SetValueOp 1320 8.5300 0.0050 0.0180 0.0065
pick 1320 10.3920 0.0060 0.0280 0.0079
Pooling 2960 4155.9810 0.0640 3.1720 1.4040
Convolution 2960 16902.8770 0.2260 13.2050 5.7104
CopyCPU2CPU 1480 8.3420 0.0030 0.0360 0.0056
log_softmax 1320 41.0940 0.0150 0.0550 0.0311
_backward_copy 1320 68.0820 0.0220 0.0760 0.0516
_backward_Convolution 2640 24451.0234 2.6370 12.5610 9.2618
_mul_scalar 1320 4.7300 0.0020 0.0230 0.0036
_backward_log_softmax 1320 22.4890 0.0090 0.0400 0.0170
mean 4120 79.1970 0.0080 0.0730 0.0192
_backward_mean 1320 14.4810 0.0080 0.1030 0.0110
_backward_FullyConnected 3960 760.2480 0.0280 0.5940 0.1920
WaitForVar 2800 25.0140 0.0060 0.0280 0.0089
broadcast_equal 1480 7.3900 0.0030 0.0110 0.0050
sgd_update 13200 130.6630 0.0020 0.0790 0.0099
_backward_Activation 5280 390.4640 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51314 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4446 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 660 2592.3711 0.3060 73.5850 3.9278
MXNDArrayCreateEx 741 25.5230 0.0310 0.0640 0.0344
MXNDArrayWaitAll 661 14357.8418 5.6130 25.0610 21.7214
MXNDArraySyncCopyToCPU 1402 11013.6602 0.1180 24.1980 7.8557
MXNDArrayGetDType 1402 0.5160 0.0000 0.0010 0.0004
MXNDArrayGetShape 3545 1.6630 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1482 12.9820 0.0070 0.0280 0.0088
MXNDArrayFree 16643 169.8470 0.0000 0.9140 0.0102
MXNet C API Calls 75127 562.4840 487.3580 562.4840 37.5630
MXNet C API Concurrency 150253 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1322 0.6250 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6610 3.4460 0.0000 0.0230 0.0005
MXNDArraySetGradState 6610 3.3660 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1322 0.5610 0.0000 0.0010 0.0004
MXAutogradBackwardEx 661 365.7870 0.5250 0.9450 0.5534
MXNDArrayGetContext 10294 4.4910 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21771 559.6750 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2644 6478.6338 0.7010 3.7500 2.4503
FullyConnected 4446 430.0630 0.0080 0.3430 0.0967
argmax 1482 29.4350 0.0050 0.0480 0.0199
_backward_pick 1322 20.9260 0.0130 0.0320 0.0158
_backward_mul_scalar 1322 5.7780 0.0030 0.0220 0.0044
DeleteVariable 55760 392.6740 0.0010 0.9010 0.0070
Flatten 1482 61.3890 0.0080 0.1120 0.0414
Activation 5928 275.7970 0.0030 1.9440 0.0465
SetValueOp 1322 8.5430 0.0050 0.0180 0.0065
pick 1322 10.4060 0.0060 0.0280 0.0079
Pooling 2964 4161.3618 0.0640 3.1720 1.4040
Convolution 2964 16925.0312 0.2260 13.2050 5.7102
CopyCPU2CPU 1482 8.4050 0.0030 0.0360 0.0057
log_softmax 1322 41.1540 0.0150 0.0550 0.0311
_backward_copy 1322 68.1860 0.0220 0.0760 0.0516
_backward_Convolution 2644 24488.0801 2.6370 12.5610 9.2618
_mul_scalar 1322 4.7380 0.0020 0.0230 0.0036
_backward_log_softmax 1322 22.5210 0.0090 0.0400 0.0170
mean 4126 79.2750 0.0080 0.0730 0.0192
_backward_mean 1322 14.5020 0.0080 0.1030 0.0110
_backward_FullyConnected 3966 761.3150 0.0280 0.5940 0.1920
WaitForVar 2804 25.0450 0.0060 0.0280 0.0089
broadcast_equal 1482 7.4000 0.0030 0.0110 0.0050
sgd_update 13220 130.8620 0.0020 0.0790 0.0099
_backward_Activation 5288 391.0310 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51388 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4452 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 661 2596.3081 0.3060 73.5850 3.9278
MXNDArrayCreateEx 742 25.5570 0.0310 0.0640 0.0344
MXNDArrayWaitAll 662 14379.7646 5.6130 25.0610 21.7217
MXNDArraySyncCopyToCPU 1404 11028.0859 0.1180 24.1980 7.8548
MXNDArrayGetDType 1404 0.5180 0.0000 0.0010 0.0004
MXNDArrayGetShape 3550 1.6660 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1484 12.9980 0.0070 0.0280 0.0088
MXNDArrayFree 16666 170.0830 0.0000 0.9140 0.0102
MXNet C API Calls 75234 562.5910 487.3580 562.5910 37.6165
MXNet C API Concurrency 150467 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1324 0.6270 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6620 3.4500 0.0000 0.0230 0.0005
MXNDArraySetGradState 6620 3.3750 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1324 0.5630 0.0000 0.0010 0.0004
MXAutogradBackwardEx 662 366.3230 0.5250 0.9450 0.5534
MXNDArrayGetContext 10308 4.4950 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21802 560.4690 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2648 6488.4009 0.7010 3.7500 2.4503
FullyConnected 4452 430.5900 0.0080 0.3430 0.0967
argmax 1484 29.4740 0.0050 0.0480 0.0199
_backward_pick 1324 20.9570 0.0130 0.0320 0.0158
_backward_mul_scalar 1324 5.7860 0.0030 0.0220 0.0044
DeleteVariable 55840 393.2180 0.0010 0.9010 0.0070
Flatten 1484 61.4710 0.0080 0.1120 0.0414
Activation 5936 276.1520 0.0030 1.9440 0.0465
SetValueOp 1324 8.5570 0.0050 0.0180 0.0065
pick 1324 10.4230 0.0060 0.0280 0.0079
Pooling 2968 4166.6968 0.0640 3.1720 1.4039
Convolution 2968 16947.2793 0.2260 13.2050 5.7100
CopyCPU2CPU 1484 8.4150 0.0030 0.0360 0.0057
log_softmax 1324 41.2140 0.0150 0.0550 0.0311
_backward_copy 1324 68.2910 0.0220 0.0760 0.0516
_backward_Convolution 2648 24525.2188 2.6370 12.5610 9.2618
_mul_scalar 1324 4.7450 0.0020 0.0230 0.0036
_backward_log_softmax 1324 22.5530 0.0090 0.0400 0.0170
mean 4132 79.3860 0.0080 0.0730 0.0192
_backward_mean 1324 14.5240 0.0080 0.1030 0.0110
_backward_FullyConnected 3972 762.3840 0.0280 0.5940 0.1919
WaitForVar 2808 25.0790 0.0060 0.0280 0.0089
broadcast_equal 1484 7.4100 0.0030 0.0110 0.0050
sgd_update 13240 131.0650 0.0020 0.0790 0.0099
_backward_Activation 5296 391.6310 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51462 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4458 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 662 2600.2290 0.3060 73.5850 3.9278
MXNDArrayCreateEx 743 25.5910 0.0310 0.0640 0.0344
MXNDArrayWaitAll 663 14401.4424 5.6130 25.0610 21.7216
MXNDArraySyncCopyToCPU 1406 11042.5830 0.1180 24.1980 7.8539
MXNDArrayGetDType 1406 0.5180 0.0000 0.0010 0.0004
MXNDArrayGetShape 3555 1.6680 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1486 13.0160 0.0070 0.0280 0.0088
MXNDArrayFree 16689 170.3110 0.0000 0.9140 0.0102
MXNet C API Calls 75341 562.6980 487.3580 562.6980 37.6700
MXNet C API Concurrency 150681 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1326 0.6280 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6630 3.4550 0.0000 0.0230 0.0005
MXNDArraySetGradState 6630 3.3800 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1326 0.5640 0.0000 0.0010 0.0004
MXAutogradBackwardEx 663 366.8580 0.5250 0.9450 0.5533
MXNDArrayGetContext 10322 4.5030 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21833 561.2650 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2652 6498.1108 0.7010 3.7500 2.4503
FullyConnected 4458 431.1810 0.0080 0.3430 0.0967
argmax 1486 29.5180 0.0050 0.0480 0.0199
_backward_pick 1326 20.9910 0.0130 0.0320 0.0158
_backward_mul_scalar 1326 5.7950 0.0030 0.0220 0.0044
DeleteVariable 55920 393.7470 0.0010 0.9010 0.0070
Flatten 1486 61.5510 0.0080 0.1120 0.0414
Activation 5944 276.5220 0.0030 1.9440 0.0465
SetValueOp 1326 8.5700 0.0050 0.0180 0.0065
pick 1326 10.4400 0.0060 0.0280 0.0079
Pooling 2972 4172.0820 0.0640 3.1720 1.4038
Convolution 2972 16969.4863 0.2260 13.2050 5.7098
CopyCPU2CPU 1486 8.4260 0.0030 0.0360 0.0057
log_softmax 1326 41.2740 0.0150 0.0550 0.0311
_backward_copy 1326 68.3930 0.0220 0.0760 0.0516
_backward_Convolution 2652 24561.9531 2.6370 12.5610 9.2617
_mul_scalar 1326 4.7520 0.0020 0.0230 0.0036
_backward_log_softmax 1326 22.5950 0.0090 0.0400 0.0170
mean 4138 79.4940 0.0080 0.0730 0.0192
_backward_mean 1326 14.5470 0.0080 0.1030 0.0110
_backward_FullyConnected 3978 763.5400 0.0280 0.5940 0.1919
WaitForVar 2812 25.1130 0.0060 0.0280 0.0089
broadcast_equal 1486 7.4190 0.0030 0.0110 0.0050
sgd_update 13260 131.2760 0.0020 0.0790 0.0099
_backward_Activation 5304 392.1940 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51536 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4464 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 663 2604.1360 0.3060 73.5850 3.9278
MXNDArrayCreateEx 744 25.6260 0.0310 0.0640 0.0344
MXNDArrayWaitAll 664 14423.1328 5.6130 25.0610 21.7216
MXNDArraySyncCopyToCPU 1408 11057.1309 0.1180 24.1980 7.8531
MXNDArrayGetDType 1408 0.5180 0.0000 0.0010 0.0004
MXNDArrayGetShape 3560 1.6700 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1488 13.0330 0.0070 0.0280 0.0088
MXNDArrayFree 16712 170.5410 0.0000 0.9140 0.0102
MXNet C API Calls 75448 562.8050 487.3580 562.8050 37.7235
MXNet C API Concurrency 150895 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1328 0.6290 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6640 3.4580 0.0000 0.0230 0.0005
MXNDArraySetGradState 6640 3.3830 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1328 0.5640 0.0000 0.0010 0.0004
MXAutogradBackwardEx 664 367.3930 0.5250 0.9450 0.5533
MXNDArrayGetContext 10336 4.5090 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21864 562.0380 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2656 6507.8599 0.7010 3.7500 2.4502
FullyConnected 4464 431.7750 0.0080 0.3430 0.0967
argmax 1488 29.5640 0.0050 0.0480 0.0199
_backward_pick 1328 21.0250 0.0130 0.0320 0.0158
_backward_mul_scalar 1328 5.8040 0.0030 0.0220 0.0044
DeleteVariable 56000 394.2810 0.0010 0.9010 0.0070
Flatten 1488 61.6320 0.0080 0.1120 0.0414
Activation 5952 276.8890 0.0030 1.9440 0.0465
SetValueOp 1328 8.5830 0.0050 0.0180 0.0065
pick 1328 10.4570 0.0060 0.0280 0.0079
Pooling 2976 4177.4351 0.0640 3.1720 1.4037
Convolution 2976 16991.6074 0.2260 13.2050 5.7095
CopyCPU2CPU 1488 8.4360 0.0030 0.0360 0.0057
log_softmax 1328 41.3330 0.0150 0.0550 0.0311
_backward_copy 1328 68.5340 0.0220 0.0760 0.0516
_backward_Convolution 2656 24598.7051 2.6370 12.5610 9.2616
_mul_scalar 1328 4.7600 0.0020 0.0230 0.0036
_backward_log_softmax 1328 22.6360 0.0090 0.0400 0.0170
mean 4144 79.5990 0.0080 0.0730 0.0192
_backward_mean 1328 14.5690 0.0080 0.1030 0.0110
_backward_FullyConnected 3984 764.7050 0.0280 0.5940 0.1919
WaitForVar 2816 25.1480 0.0060 0.0280 0.0089
broadcast_equal 1488 7.4300 0.0030 0.0110 0.0050
sgd_update 13280 131.4860 0.0020 0.0790 0.0099
_backward_Activation 5312 392.7510 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51610 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4470 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 664 2608.0471 0.3060 73.5850 3.9278
MXNDArrayCreateEx 745 25.6600 0.0310 0.0640 0.0344
MXNDArrayWaitAll 665 14444.9785 5.6130 25.0610 21.7218
MXNDArraySyncCopyToCPU 1410 11071.3848 0.1180 24.1980 7.8520
MXNDArrayGetDType 1410 0.5190 0.0000 0.0010 0.0004
MXNDArrayGetShape 3565 1.6700 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1490 13.0510 0.0070 0.0280 0.0088
MXNDArrayFree 16735 170.7700 0.0000 0.9140 0.0102
MXNet C API Calls 75555 562.9120 487.3580 562.9120 37.7770
MXNet C API Concurrency 151109 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1330 0.6310 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6650 3.4660 0.0000 0.0230 0.0005
MXNDArraySetGradState 6650 3.3900 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1330 0.5660 0.0000 0.0010 0.0004
MXAutogradBackwardEx 665 367.9690 0.5250 0.9450 0.5533
MXNDArrayGetContext 10350 4.5140 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21895 562.8340 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2660 6517.5718 0.7010 3.7500 2.4502
FullyConnected 4470 432.3020 0.0080 0.3430 0.0967
argmax 1490 29.6010 0.0050 0.0480 0.0199
_backward_pick 1330 21.0560 0.0130 0.0320 0.0158
_backward_mul_scalar 1330 5.8120 0.0030 0.0220 0.0044
DeleteVariable 56080 394.7980 0.0010 0.9010 0.0070
Flatten 1490 61.7130 0.0080 0.1120 0.0414
Activation 5960 277.2440 0.0030 1.9440 0.0465
SetValueOp 1330 8.5970 0.0050 0.0180 0.0065
pick 1330 10.4730 0.0060 0.0280 0.0079
Pooling 2980 4182.8179 0.0640 3.1720 1.4036
Convolution 2980 17013.8184 0.2260 13.2050 5.7093
CopyCPU2CPU 1490 8.4460 0.0030 0.0360 0.0057
log_softmax 1330 41.3930 0.0150 0.0550 0.0311
_backward_copy 1330 68.6380 0.0220 0.0760 0.0516
_backward_Convolution 2660 24635.7871 2.6370 12.5610 9.2616
_mul_scalar 1330 4.7670 0.0020 0.0230 0.0036
_backward_log_softmax 1330 22.6680 0.0090 0.0400 0.0170
mean 4150 79.7170 0.0080 0.0730 0.0192
_backward_mean 1330 14.5910 0.0080 0.1030 0.0110
_backward_FullyConnected 3990 765.7740 0.0280 0.5940 0.1919
WaitForVar 2820 25.1830 0.0060 0.0280 0.0089
broadcast_equal 1490 7.4400 0.0030 0.0110 0.0050
sgd_update 13300 131.6860 0.0020 0.0790 0.0099
_backward_Activation 5320 393.3040 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51684 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4476 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 665 2611.9600 0.3060 73.5850 3.9278
MXNDArrayCreateEx 746 25.6950 0.0310 0.0640 0.0344
MXNDArrayWaitAll 666 14466.8281 5.6130 25.0610 21.7220
MXNDArraySyncCopyToCPU 1412 11085.8906 0.1180 24.1980 7.8512
MXNDArrayGetDType 1412 0.5190 0.0000 0.0010 0.0004
MXNDArrayGetShape 3570 1.6730 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1492 13.0680 0.0070 0.0280 0.0088
MXNDArrayFree 16758 170.9960 0.0000 0.9140 0.0102
MXNet C API Calls 75662 563.0190 487.3580 563.0190 37.8305
MXNet C API Concurrency 151323 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1332 0.6310 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6660 3.4720 0.0000 0.0230 0.0005
MXNDArraySetGradState 6660 3.3960 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1332 0.5670 0.0000 0.0010 0.0004
MXAutogradBackwardEx 666 368.5040 0.5250 0.9450 0.5533
MXNDArrayGetContext 10364 4.5190 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21926 563.6340 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2664 6527.3911 0.7010 3.7500 2.4502
FullyConnected 4476 432.8310 0.0080 0.3430 0.0967
argmax 1492 29.6390 0.0050 0.0480 0.0199
_backward_pick 1332 21.0880 0.0130 0.0320 0.0158
_backward_mul_scalar 1332 5.8210 0.0030 0.0220 0.0044
DeleteVariable 56160 395.3340 0.0010 0.9010 0.0070
Flatten 1492 61.7940 0.0080 0.1120 0.0414
Activation 5968 277.5980 0.0030 1.9440 0.0465
SetValueOp 1332 8.6110 0.0050 0.0180 0.0065
pick 1332 10.4870 0.0060 0.0280 0.0079
Pooling 2984 4188.2002 0.0640 3.1720 1.4036
Convolution 2984 17036.0625 0.2260 13.2050 5.7091
CopyCPU2CPU 1492 8.4560 0.0030 0.0360 0.0057
log_softmax 1332 41.4520 0.0150 0.0550 0.0311
_backward_copy 1332 68.7430 0.0220 0.0760 0.0516
_backward_Convolution 2664 24672.8516 2.6370 12.5610 9.2616
_mul_scalar 1332 4.7740 0.0020 0.0230 0.0036
_backward_log_softmax 1332 22.7010 0.0090 0.0400 0.0170
mean 4156 79.8320 0.0080 0.0730 0.0192
_backward_mean 1332 14.6120 0.0080 0.1030 0.0110
_backward_FullyConnected 3996 766.8480 0.0280 0.5940 0.1919
WaitForVar 2824 25.2180 0.0060 0.0280 0.0089
broadcast_equal 1492 7.4500 0.0030 0.0110 0.0050
sgd_update 13320 131.8890 0.0020 0.0790 0.0099
_backward_Activation 5328 393.8590 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51758 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4482 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 666 2615.8760 0.3060 73.5850 3.9277
MXNDArrayCreateEx 747 25.7290 0.0310 0.0640 0.0344
MXNDArrayWaitAll 667 14488.5684 5.6130 25.0610 21.7220
MXNDArraySyncCopyToCPU 1414 11100.4121 0.1180 24.1980 7.8504
MXNDArrayGetDType 1414 0.5200 0.0000 0.0010 0.0004
MXNDArrayGetShape 3575 1.6750 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1494 13.0860 0.0070 0.0280 0.0088
MXNDArrayFree 16781 171.2240 0.0000 0.9140 0.0102
MXNet C API Calls 75769 563.1260 487.3580 563.1260 37.8840
MXNet C API Concurrency 151537 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1334 0.6320 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6670 3.4750 0.0000 0.0230 0.0005
MXNDArraySetGradState 6670 3.4000 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1334 0.5680 0.0000 0.0010 0.0004
MXAutogradBackwardEx 667 369.0420 0.5250 0.9450 0.5533
MXNDArrayGetContext 10378 4.5270 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21957 564.4300 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2668 6537.1602 0.7010 3.7500 2.4502
FullyConnected 4482 433.4650 0.0080 0.3430 0.0967
argmax 1494 29.6770 0.0050 0.0480 0.0199
_backward_pick 1334 21.1200 0.0130 0.0320 0.0158
_backward_mul_scalar 1334 5.8300 0.0030 0.0220 0.0044
DeleteVariable 56240 395.8860 0.0010 0.9010 0.0070
Flatten 1494 61.8770 0.0080 0.1120 0.0414
Activation 5976 277.9670 0.0030 1.9440 0.0465
SetValueOp 1334 8.6240 0.0050 0.0180 0.0065
pick 1334 10.5010 0.0060 0.0280 0.0079
Pooling 2988 4193.5508 0.0640 3.1720 1.4035
Convolution 2988 17058.2793 0.2260 13.2050 5.7089
CopyCPU2CPU 1494 8.4660 0.0030 0.0360 0.0057
log_softmax 1334 41.5130 0.0150 0.0550 0.0311
_backward_copy 1334 68.8450 0.0220 0.0760 0.0516
_backward_Convolution 2668 24709.6523 2.6370 12.5610 9.2615
_mul_scalar 1334 4.7810 0.0020 0.0230 0.0036
_backward_log_softmax 1334 22.7440 0.0090 0.0400 0.0170
mean 4162 79.9380 0.0080 0.0730 0.0192
_backward_mean 1334 14.6340 0.0080 0.1030 0.0110
_backward_FullyConnected 4002 767.9940 0.0280 0.5940 0.1919
WaitForVar 2828 25.2530 0.0060 0.0280 0.0089
broadcast_equal 1494 7.4610 0.0030 0.0110 0.0050
sgd_update 13340 132.1260 0.0020 0.0790 0.0099
_backward_Activation 5336 394.4340 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51832 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4488 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 667 2619.7891 0.3060 73.5850 3.9277
MXNDArrayCreateEx 748 25.7640 0.0310 0.0640 0.0344
MXNDArrayWaitAll 668 14510.2910 5.6130 25.0610 21.7220
MXNDArraySyncCopyToCPU 1416 11114.8359 0.1180 24.1980 7.8495
MXNDArrayGetDType 1416 0.5200 0.0000 0.0010 0.0004
MXNDArrayGetShape 3580 1.6770 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1496 13.1040 0.0070 0.0280 0.0088
MXNDArrayFree 16804 171.4540 0.0000 0.9140 0.0102
MXNet C API Calls 75876 563.2330 487.3580 563.2330 37.9375
MXNet C API Concurrency 151751 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1336 0.6320 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6680 3.4800 0.0000 0.0230 0.0005
MXNDArraySetGradState 6680 3.4030 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1336 0.5690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 668 369.5790 0.5250 0.9450 0.5533
MXNDArrayGetContext 10392 4.5310 0.0000 0.0070 0.0004
MXImperativeInvokeEx 21988 565.2070 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2672 6546.9648 0.7010 3.7500 2.4502
FullyConnected 4488 434.0430 0.0080 0.3430 0.0967
argmax 1496 29.7160 0.0050 0.0480 0.0199
_backward_pick 1336 21.1520 0.0130 0.0320 0.0158
_backward_mul_scalar 1336 5.8380 0.0030 0.0220 0.0044
DeleteVariable 56320 396.4300 0.0010 0.9010 0.0070
Flatten 1496 61.9580 0.0080 0.1120 0.0414
Activation 5984 278.3390 0.0030 1.9440 0.0465
SetValueOp 1336 8.6370 0.0050 0.0180 0.0065
pick 1336 10.5180 0.0060 0.0280 0.0079
Pooling 2992 4198.8960 0.0640 3.1720 1.4034
Convolution 2992 17080.3711 0.2260 13.2050 5.7087
CopyCPU2CPU 1496 8.4760 0.0030 0.0360 0.0057
log_softmax 1336 41.5820 0.0150 0.0550 0.0311
_backward_copy 1336 68.9470 0.0220 0.0760 0.0516
_backward_Convolution 2672 24746.4102 2.6370 12.5610 9.2614
_mul_scalar 1336 4.7880 0.0020 0.0230 0.0036
_backward_log_softmax 1336 22.7750 0.0090 0.0400 0.0170
mean 4168 80.0430 0.0080 0.0730 0.0192
_backward_mean 1336 14.6560 0.0080 0.1030 0.0110
_backward_FullyConnected 4008 769.1550 0.0280 0.5940 0.1919
WaitForVar 2832 25.2880 0.0060 0.0280 0.0089
broadcast_equal 1496 7.4710 0.0030 0.0110 0.0050
sgd_update 13360 132.3320 0.0020 0.0790 0.0099
_backward_Activation 5344 394.9850 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51906 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4494 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 668 2623.7000 0.3060 73.5850 3.9277
MXNDArrayCreateEx 749 25.7990 0.0310 0.0640 0.0344
MXNDArrayWaitAll 669 14532.1230 5.6130 25.0610 21.7222
MXNDArraySyncCopyToCPU 1418 11129.0469 0.1180 24.1980 7.8484
MXNDArrayGetDType 1418 0.5200 0.0000 0.0010 0.0004
MXNDArrayGetShape 3585 1.6780 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1498 13.1210 0.0070 0.0280 0.0088
MXNDArrayFree 16827 171.6840 0.0000 0.9140 0.0102
MXNet C API Calls 75983 563.3400 487.3580 563.3400 37.9910
MXNet C API Concurrency 151965 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1338 0.6330 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6690 3.4840 0.0000 0.0230 0.0005
MXNDArraySetGradState 6690 3.4080 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1338 0.5690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 669 370.1370 0.5250 0.9450 0.5533
MXNDArrayGetContext 10406 4.5370 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22019 566.0060 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2676 6556.6060 0.7010 3.7500 2.4502
FullyConnected 4494 434.5660 0.0080 0.3430 0.0967
argmax 1498 29.7550 0.0050 0.0480 0.0199
_backward_pick 1338 21.1840 0.0130 0.0320 0.0158
_backward_mul_scalar 1338 5.8470 0.0030 0.0220 0.0044
DeleteVariable 56400 396.9620 0.0010 0.9010 0.0070
Flatten 1498 62.0400 0.0080 0.1120 0.0414
Activation 5992 278.6910 0.0030 1.9440 0.0465
SetValueOp 1338 8.6500 0.0050 0.0180 0.0065
pick 1338 10.5340 0.0060 0.0280 0.0079
Pooling 2996 4204.2158 0.0640 3.1720 1.4033
Convolution 2996 17102.5820 0.2260 13.2050 5.7085
CopyCPU2CPU 1498 8.4860 0.0030 0.0360 0.0057
log_softmax 1338 41.6420 0.0150 0.0550 0.0311
_backward_copy 1338 69.0520 0.0220 0.0760 0.0516
_backward_Convolution 2676 24783.4512 2.6370 12.5610 9.2614
_mul_scalar 1338 4.7950 0.0020 0.0230 0.0036
_backward_log_softmax 1338 22.8070 0.0090 0.0400 0.0170
mean 4174 80.1580 0.0080 0.0730 0.0192
_backward_mean 1338 14.6780 0.0080 0.1030 0.0110
_backward_FullyConnected 4014 770.2770 0.0280 0.5940 0.1919
WaitForVar 2836 25.3220 0.0060 0.0280 0.0089
broadcast_equal 1498 7.4820 0.0030 0.0110 0.0050
sgd_update 13380 132.5290 0.0020 0.0790 0.0099
_backward_Activation 5352 395.5460 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 51980 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4500 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 669 2627.6240 0.3060 73.5850 3.9277
MXNDArrayCreateEx 750 25.8330 0.0310 0.0640 0.0344
MXNDArrayWaitAll 670 14553.9912 5.6130 25.0610 21.7224
MXNDArraySyncCopyToCPU 1420 11143.5059 0.1180 24.1980 7.8475
MXNDArrayGetDType 1420 0.5200 0.0000 0.0010 0.0004
MXNDArrayGetShape 3590 1.6790 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1500 13.1380 0.0070 0.0280 0.0088
MXNDArrayFree 16850 171.9120 0.0000 0.9140 0.0102
MXNet C API Calls 76090 563.4470 487.3580 563.4470 38.0445
MXNet C API Concurrency 152179 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1340 0.6340 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6700 3.4890 0.0000 0.0230 0.0005
MXNDArraySetGradState 6700 3.4130 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1340 0.5700 0.0000 0.0010 0.0004
MXAutogradBackwardEx 670 370.7090 0.5250 0.9450 0.5533
MXNDArrayGetContext 10420 4.5440 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22050 566.7800 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2680 6566.3701 0.7010 3.7500 2.4501
FullyConnected 4500 435.1180 0.0080 0.3430 0.0967
argmax 1500 29.7940 0.0050 0.0480 0.0199
_backward_pick 1340 21.2140 0.0130 0.0320 0.0158
_backward_mul_scalar 1340 5.8550 0.0030 0.0220 0.0044
DeleteVariable 56480 397.4940 0.0010 0.9010 0.0070
Flatten 1500 62.1220 0.0080 0.1120 0.0414
Activation 6000 279.0450 0.0030 1.9440 0.0465
SetValueOp 1340 8.6640 0.0050 0.0180 0.0065
pick 1340 10.5490 0.0060 0.0280 0.0079
Pooling 3000 4209.6519 0.0640 3.1720 1.4032
Convolution 3000 17124.8027 0.2260 13.2050 5.7083
CopyCPU2CPU 1500 8.4960 0.0030 0.0360 0.0057
log_softmax 1340 41.7020 0.0150 0.0550 0.0311
_backward_copy 1340 69.1570 0.0220 0.0760 0.0516
_backward_Convolution 2680 24820.5195 2.6370 12.5610 9.2614
_mul_scalar 1340 4.8010 0.0020 0.0230 0.0036
_backward_log_softmax 1340 22.8400 0.0090 0.0400 0.0170
mean 4180 80.2810 0.0080 0.0730 0.0192
_backward_mean 1340 14.7000 0.0080 0.1030 0.0110
_backward_FullyConnected 4020 771.3510 0.0280 0.5940 0.1919
WaitForVar 2840 25.3570 0.0060 0.0280 0.0089
broadcast_equal 1500 7.4920 0.0030 0.0110 0.0050
sgd_update 13400 132.7260 0.0020 0.0790 0.0099
_backward_Activation 5360 396.1000 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52054 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4506 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 670 2631.5381 0.3060 73.5850 3.9277
MXNDArrayCreateEx 751 25.8680 0.0310 0.0640 0.0344
MXNDArrayWaitAll 671 14575.6611 5.6130 25.0610 21.7223
MXNDArraySyncCopyToCPU 1422 11157.9873 0.1180 24.1980 7.8467
MXNDArrayGetDType 1422 0.5210 0.0000 0.0010 0.0004
MXNDArrayGetShape 3595 1.6810 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1502 13.1560 0.0070 0.0280 0.0088
MXNDArrayFree 16873 172.1620 0.0000 0.9140 0.0102
MXNet C API Calls 76197 563.5540 487.3580 563.5540 38.0980
MXNet C API Concurrency 152393 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1342 0.6340 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6710 3.4930 0.0000 0.0230 0.0005
MXNDArraySetGradState 6710 3.4180 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1342 0.5700 0.0000 0.0010 0.0004
MXAutogradBackwardEx 671 371.2560 0.5250 0.9450 0.5533
MXNDArrayGetContext 10434 4.5490 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22081 567.5520 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2684 6575.9990 0.7010 3.7500 2.4501
FullyConnected 4506 435.6990 0.0080 0.3430 0.0967
argmax 1502 29.8370 0.0050 0.0480 0.0199
_backward_pick 1342 21.2460 0.0130 0.0320 0.0158
_backward_mul_scalar 1342 5.8640 0.0030 0.0220 0.0044
DeleteVariable 56560 398.0610 0.0010 0.9010 0.0070
Flatten 1502 62.2060 0.0080 0.1120 0.0414
Activation 6008 279.4140 0.0030 1.9440 0.0465
SetValueOp 1342 8.6780 0.0050 0.0180 0.0065
pick 1342 10.5660 0.0060 0.0280 0.0079
Pooling 3004 4215.0352 0.0640 3.1720 1.4031
Convolution 3004 17146.9824 0.2260 13.2050 5.7081
CopyCPU2CPU 1502 8.5050 0.0030 0.0360 0.0057
log_softmax 1342 41.7690 0.0150 0.0550 0.0311
_backward_copy 1342 69.2580 0.0220 0.0760 0.0516
_backward_Convolution 2684 24857.2773 2.6370 12.5610 9.2613
_mul_scalar 1342 4.8080 0.0020 0.0230 0.0036
_backward_log_softmax 1342 22.8730 0.0090 0.0400 0.0170
mean 4186 80.3890 0.0080 0.0730 0.0192
_backward_mean 1342 14.7220 0.0080 0.1030 0.0110
_backward_FullyConnected 4026 772.5230 0.0280 0.5940 0.1919
WaitForVar 2844 25.3920 0.0060 0.0280 0.0089
broadcast_equal 1502 7.5020 0.0030 0.0110 0.0050
sgd_update 13420 132.9390 0.0020 0.0790 0.0099
_backward_Activation 5368 396.6910 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52128 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4512 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 671 2635.4900 0.3060 73.5850 3.9277
MXNDArrayCreateEx 752 25.9030 0.0310 0.0640 0.0344
MXNDArrayWaitAll 672 14597.4170 5.6130 25.0610 21.7223
MXNDArraySyncCopyToCPU 1424 11172.3965 0.1180 24.1980 7.8458
MXNDArrayGetDType 1424 0.5210 0.0000 0.0010 0.0004
MXNDArrayGetShape 3600 1.6820 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1504 13.1740 0.0070 0.0280 0.0088
MXNDArrayFree 16896 172.3930 0.0000 0.9140 0.0102
MXNet C API Calls 76304 563.6610 487.3580 563.6610 38.1515
MXNet C API Concurrency 152607 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1344 0.6350 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6720 3.4970 0.0000 0.0230 0.0005
MXNDArraySetGradState 6720 3.4230 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1344 0.5710 0.0000 0.0010 0.0004
MXAutogradBackwardEx 672 371.8020 0.5250 0.9450 0.5533
MXNDArrayGetContext 10448 4.5560 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22112 568.3370 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2688 6585.7710 0.7010 3.7500 2.4501
FullyConnected 4512 436.3090 0.0080 0.3430 0.0967
argmax 1504 29.8750 0.0050 0.0480 0.0199
_backward_pick 1344 21.2770 0.0130 0.0320 0.0158
_backward_mul_scalar 1344 5.8720 0.0030 0.0220 0.0044
DeleteVariable 56640 398.5900 0.0010 0.9010 0.0070
Flatten 1504 62.2940 0.0080 0.1120 0.0414
Activation 6016 279.7880 0.0030 1.9440 0.0465
SetValueOp 1344 8.6920 0.0050 0.0180 0.0065
pick 1344 10.5820 0.0060 0.0280 0.0079
Pooling 3008 4220.3672 0.0640 3.1720 1.4030
Convolution 3008 17169.0723 0.2260 13.2050 5.7078
CopyCPU2CPU 1504 8.5150 0.0030 0.0360 0.0057
log_softmax 1344 41.8440 0.0150 0.0550 0.0311
_backward_copy 1344 69.3600 0.0220 0.0760 0.0516
_backward_Convolution 2688 24894.0469 2.6370 12.5610 9.2612
_mul_scalar 1344 4.8150 0.0020 0.0230 0.0036
_backward_log_softmax 1344 22.9060 0.0090 0.0400 0.0170
mean 4192 80.5120 0.0080 0.0730 0.0192
_backward_mean 1344 14.7460 0.0080 0.1030 0.0110
_backward_FullyConnected 4032 773.6860 0.0280 0.5940 0.1919
WaitForVar 2848 25.4290 0.0060 0.0280 0.0089
broadcast_equal 1504 7.5110 0.0030 0.0110 0.0050
sgd_update 13440 133.1240 0.0020 0.0790 0.0099
_backward_Activation 5376 397.3380 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52202 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4518 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 672 2639.4270 0.3060 73.5850 3.9277
MXNDArrayCreateEx 753 25.9370 0.0310 0.0640 0.0344
MXNDArrayWaitAll 673 14619.2275 5.6130 25.0610 21.7225
MXNDArraySyncCopyToCPU 1426 11186.8066 0.1180 24.1980 7.8449
MXNDArrayGetDType 1426 0.5210 0.0000 0.0010 0.0004
MXNDArrayGetShape 3605 1.6860 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1506 13.1920 0.0070 0.0280 0.0088
MXNDArrayFree 16919 172.6360 0.0000 0.9140 0.0102
MXNet C API Calls 76411 563.7680 487.3580 563.7680 38.2050
MXNet C API Concurrency 152821 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1346 0.6370 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6730 3.5000 0.0000 0.0230 0.0005
MXNDArraySetGradState 6730 3.4290 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1346 0.5730 0.0000 0.0010 0.0004
MXAutogradBackwardEx 673 372.3460 0.5250 0.9450 0.5533
MXNDArrayGetContext 10462 4.5620 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22143 569.1270 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2692 6595.4590 0.7010 3.7500 2.4500
FullyConnected 4518 436.8380 0.0080 0.3430 0.0967
argmax 1506 29.9130 0.0050 0.0480 0.0199
_backward_pick 1346 21.3080 0.0130 0.0320 0.0158
_backward_mul_scalar 1346 5.8810 0.0030 0.0220 0.0044
DeleteVariable 56720 399.1370 0.0010 0.9010 0.0070
Flatten 1506 62.3790 0.0080 0.1120 0.0414
Activation 6024 280.1440 0.0030 1.9440 0.0465
SetValueOp 1346 8.7040 0.0050 0.0180 0.0065
pick 1346 10.5980 0.0060 0.0280 0.0079
Pooling 3012 4225.7578 0.0640 3.1720 1.4030
Convolution 3012 17191.2676 0.2260 13.2050 5.7076
CopyCPU2CPU 1506 8.5250 0.0030 0.0360 0.0057
log_softmax 1346 41.9050 0.0150 0.0550 0.0311
_backward_copy 1346 69.4650 0.0220 0.0760 0.0516
_backward_Convolution 2692 24931.0859 2.6370 12.5610 9.2612
_mul_scalar 1346 4.8220 0.0020 0.0230 0.0036
_backward_log_softmax 1346 22.9370 0.0090 0.0400 0.0170
mean 4198 80.6220 0.0080 0.0730 0.0192
_backward_mean 1346 14.7680 0.0080 0.1030 0.0110
_backward_FullyConnected 4038 774.7650 0.0280 0.5940 0.1919
WaitForVar 2852 25.4650 0.0060 0.0280 0.0089
broadcast_equal 1506 7.5210 0.0030 0.0110 0.0050
sgd_update 13460 133.2780 0.0020 0.0790 0.0099
_backward_Activation 5384 397.9660 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52276 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4524 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 673 2643.3501 0.3060 73.5850 3.9277
MXNDArrayCreateEx 754 25.9720 0.0310 0.0640 0.0344
MXNDArrayWaitAll 674 14641.1357 5.6130 25.0610 21.7228
MXNDArraySyncCopyToCPU 1428 11201.2598 0.1180 24.1980 7.8440
MXNDArrayGetDType 1428 0.5220 0.0000 0.0010 0.0004
MXNDArrayGetShape 3610 1.6900 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1508 13.2090 0.0070 0.0280 0.0088
MXNDArrayFree 16942 172.8690 0.0000 0.9140 0.0102
MXNet C API Calls 76518 563.8750 487.3580 563.8750 38.2585
MXNet C API Concurrency 153035 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1348 0.6380 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6740 3.5070 0.0000 0.0230 0.0005
MXNDArraySetGradState 6740 3.4340 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1348 0.5740 0.0000 0.0010 0.0004
MXAutogradBackwardEx 674 372.8830 0.5250 0.9450 0.5532
MXNDArrayGetContext 10476 4.5710 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22174 569.8990 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2696 6605.1992 0.7010 3.7500 2.4500
FullyConnected 4524 437.3990 0.0080 0.3430 0.0967
argmax 1508 29.9510 0.0050 0.0480 0.0199
_backward_pick 1348 21.3410 0.0130 0.0320 0.0158
_backward_mul_scalar 1348 5.8900 0.0030 0.0220 0.0044
DeleteVariable 56800 399.7210 0.0010 0.9010 0.0070
Flatten 1508 62.4640 0.0080 0.1120 0.0414
Activation 6032 280.4990 0.0030 1.9440 0.0465
SetValueOp 1348 8.7180 0.0050 0.0180 0.0065
pick 1348 10.6130 0.0060 0.0280 0.0079
Pooling 3016 4231.0801 0.0640 3.1720 1.4029
Convolution 3016 17213.5449 0.2260 13.2050 5.7074
CopyCPU2CPU 1508 8.5350 0.0030 0.0360 0.0057
log_softmax 1348 41.9660 0.0150 0.0550 0.0311
_backward_copy 1348 69.5690 0.0220 0.0760 0.0516
_backward_Convolution 2696 24968.2148 2.6370 12.5610 9.2612
_mul_scalar 1348 4.8300 0.0020 0.0230 0.0036
_backward_log_softmax 1348 22.9680 0.0090 0.0400 0.0170
mean 4204 80.7360 0.0080 0.0730 0.0192
_backward_mean 1348 14.7900 0.0080 0.1030 0.0110
_backward_FullyConnected 4044 775.8440 0.0280 0.5940 0.1919
WaitForVar 2856 25.4950 0.0060 0.0280 0.0089
broadcast_equal 1508 7.5320 0.0030 0.0110 0.0050
sgd_update 13480 133.4770 0.0020 0.0790 0.0099
_backward_Activation 5392 398.5100 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52350 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4530 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 674 2647.2629 0.3060 73.5850 3.9277
MXNDArrayCreateEx 755 26.0060 0.0310 0.0640 0.0344
MXNDArrayWaitAll 675 14662.8877 5.6130 25.0610 21.7228
MXNDArraySyncCopyToCPU 1430 11215.7959 0.1180 24.1980 7.8432
MXNDArrayGetDType 1430 0.5220 0.0000 0.0010 0.0004
MXNDArrayGetShape 3615 1.6920 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1510 13.2260 0.0070 0.0280 0.0088
MXNDArrayFree 16965 173.1010 0.0000 0.9140 0.0102
MXNet C API Calls 76625 563.9820 487.3580 563.9820 38.3120
MXNet C API Concurrency 153249 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1350 0.6380 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6750 3.5120 0.0000 0.0230 0.0005
MXNDArraySetGradState 6750 3.4400 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1350 0.5740 0.0000 0.0010 0.0004
MXAutogradBackwardEx 675 373.4260 0.5250 0.9450 0.5532
MXNDArrayGetContext 10490 4.5770 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22205 570.6850 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2700 6615.0020 0.7010 3.7500 2.4500
FullyConnected 4530 437.9960 0.0080 0.3430 0.0967
argmax 1510 29.9890 0.0050 0.0480 0.0199
_backward_pick 1350 21.3740 0.0130 0.0320 0.0158
_backward_mul_scalar 1350 5.8990 0.0030 0.0220 0.0044
DeleteVariable 56880 400.2630 0.0010 0.9010 0.0070
Flatten 1510 62.5490 0.0080 0.1120 0.0414
Activation 6040 280.8730 0.0030 1.9440 0.0465
SetValueOp 1350 8.7330 0.0050 0.0180 0.0065
pick 1350 10.6290 0.0060 0.0280 0.0079
Pooling 3020 4236.4751 0.0640 3.1720 1.4028
Convolution 3020 17235.7871 0.2260 13.2050 5.7072
CopyCPU2CPU 1510 8.5450 0.0030 0.0360 0.0057
log_softmax 1350 42.0310 0.0150 0.0550 0.0311
_backward_copy 1350 69.6710 0.0220 0.0760 0.0516
_backward_Convolution 2700 25005.0078 2.6370 12.5610 9.2611
_mul_scalar 1350 4.8370 0.0020 0.0230 0.0036
_backward_log_softmax 1350 23.0050 0.0090 0.0400 0.0170
mean 4210 80.8410 0.0080 0.0730 0.0192
_backward_mean 1350 14.8130 0.0080 0.1030 0.0110
_backward_FullyConnected 4050 777.0420 0.0280 0.5940 0.1919
WaitForVar 2860 25.5290 0.0060 0.0280 0.0089
broadcast_equal 1510 7.5430 0.0030 0.0110 0.0050
sgd_update 13500 133.6360 0.0020 0.0790 0.0099
_backward_Activation 5400 399.1410 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52424 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4536 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 675 2651.1750 0.3060 73.5850 3.9277
MXNDArrayCreateEx 756 26.0390 0.0310 0.0640 0.0344
MXNDArrayWaitAll 676 14684.5986 5.6130 25.0610 21.7228
MXNDArraySyncCopyToCPU 1432 11230.2715 0.1180 24.1980 7.8424
MXNDArrayGetDType 1432 0.5220 0.0000 0.0010 0.0004
MXNDArrayGetShape 3620 1.6960 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1512 13.2430 0.0070 0.0280 0.0088
MXNDArrayFree 16988 173.3290 0.0000 0.9140 0.0102
MXNet C API Calls 76732 564.0890 487.3580 564.0890 38.3655
MXNet C API Concurrency 153463 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1352 0.6400 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6760 3.5170 0.0000 0.0230 0.0005
MXNDArraySetGradState 6760 3.4490 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1352 0.5760 0.0000 0.0010 0.0004
MXAutogradBackwardEx 676 373.9640 0.5250 0.9450 0.5532
MXNDArrayGetContext 10504 4.5850 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22236 571.4580 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2704 6624.7422 0.7010 3.7500 2.4500
FullyConnected 4536 438.5870 0.0080 0.3430 0.0967
argmax 1512 30.0310 0.0050 0.0480 0.0199
_backward_pick 1352 21.4100 0.0130 0.0320 0.0158
_backward_mul_scalar 1352 5.9080 0.0030 0.0220 0.0044
DeleteVariable 56960 400.7850 0.0010 0.9010 0.0070
Flatten 1512 62.6300 0.0080 0.1120 0.0414
Activation 6048 281.2400 0.0030 1.9440 0.0465
SetValueOp 1352 8.7470 0.0050 0.0180 0.0065
pick 1352 10.6450 0.0060 0.0280 0.0079
Pooling 3024 4241.8262 0.0640 3.1720 1.4027
Convolution 3024 17257.9082 0.2260 13.2050 5.7070
CopyCPU2CPU 1512 8.5560 0.0030 0.0360 0.0057
log_softmax 1352 42.0970 0.0150 0.0550 0.0311
_backward_copy 1352 69.7740 0.0220 0.0760 0.0516
_backward_Convolution 2704 25041.7891 2.6370 12.5610 9.2610
_mul_scalar 1352 4.8440 0.0020 0.0230 0.0036
_backward_log_softmax 1352 23.0410 0.0090 0.0400 0.0170
mean 4216 80.9490 0.0080 0.0730 0.0192
_backward_mean 1352 14.8360 0.0080 0.1030 0.0110
_backward_FullyConnected 4056 778.2600 0.0280 0.5940 0.1919
WaitForVar 2864 25.5640 0.0060 0.0280 0.0089
broadcast_equal 1512 7.5530 0.0030 0.0110 0.0050
sgd_update 13520 133.8570 0.0020 0.0790 0.0099
_backward_Activation 5408 399.7070 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52498 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4542 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 676 2655.1069 0.3060 73.5850 3.9277
MXNDArrayCreateEx 757 26.0740 0.0310 0.0640 0.0344
MXNDArrayWaitAll 677 14706.5225 5.6130 25.0610 21.7231
MXNDArraySyncCopyToCPU 1434 11244.6553 0.1180 24.1980 7.8415
MXNDArrayGetDType 1434 0.5230 0.0000 0.0010 0.0004
MXNDArrayGetShape 3625 1.6980 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1514 13.2610 0.0070 0.0280 0.0088
MXNDArrayFree 17011 173.5560 0.0000 0.9140 0.0102
MXNet C API Calls 76839 564.1960 487.3580 564.1960 38.4190
MXNet C API Concurrency 153677 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1354 0.6400 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6770 3.5230 0.0000 0.0230 0.0005
MXNDArraySetGradState 6770 3.4530 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1354 0.5770 0.0000 0.0010 0.0004
MXAutogradBackwardEx 677 374.5290 0.5250 0.9450 0.5532
MXNDArrayGetContext 10518 4.5900 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22267 572.2410 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2708 6634.6299 0.7010 3.7500 2.4500
FullyConnected 4542 439.1190 0.0080 0.3430 0.0967
argmax 1514 30.0700 0.0050 0.0480 0.0199
_backward_pick 1354 21.4410 0.0130 0.0320 0.0158
_backward_mul_scalar 1354 5.9160 0.0030 0.0220 0.0044
DeleteVariable 57040 401.3760 0.0010 0.9010 0.0070
Flatten 1514 62.7170 0.0080 0.1120 0.0414
Activation 6056 281.5980 0.0030 1.9440 0.0465
SetValueOp 1354 8.7600 0.0050 0.0180 0.0065
pick 1354 10.6600 0.0060 0.0280 0.0079
Pooling 3028 4247.2632 0.0640 3.1720 1.4027
Convolution 3028 17280.1191 0.2260 13.2050 5.7068
CopyCPU2CPU 1514 8.5670 0.0030 0.0360 0.0057
log_softmax 1354 42.1570 0.0150 0.0550 0.0311
_backward_copy 1354 69.8770 0.0220 0.0760 0.0516
_backward_Convolution 2708 25078.8359 2.6370 12.5610 9.2610
_mul_scalar 1354 4.8510 0.0020 0.0230 0.0036
_backward_log_softmax 1354 23.0730 0.0090 0.0400 0.0170
mean 4222 81.0640 0.0080 0.0730 0.0192
_backward_mean 1354 14.8580 0.0080 0.1030 0.0110
_backward_FullyConnected 4062 779.3400 0.0280 0.5940 0.1919
WaitForVar 2868 25.5990 0.0060 0.0280 0.0089
broadcast_equal 1514 7.5630 0.0030 0.0110 0.0050
sgd_update 13540 134.0570 0.0020 0.0790 0.0099
_backward_Activation 5416 400.2870 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52572 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4548 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 677 2659.0181 0.3060 73.5850 3.9276
MXNDArrayCreateEx 758 26.1080 0.0310 0.0640 0.0344
MXNDArrayWaitAll 678 14728.4463 5.6130 25.0610 21.7234
MXNDArraySyncCopyToCPU 1436 11259.1152 0.1180 24.1980 7.8406
MXNDArrayGetDType 1436 0.5230 0.0000 0.0010 0.0004
MXNDArrayGetShape 3630 1.6990 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1516 13.2790 0.0070 0.0280 0.0088
MXNDArrayFree 17034 173.7900 0.0000 0.9140 0.0102
MXNet C API Calls 76946 564.3030 487.3580 564.3030 38.4725
MXNet C API Concurrency 153891 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1356 0.6400 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6780 3.5270 0.0000 0.0230 0.0005
MXNDArraySetGradState 6780 3.4600 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1356 0.5770 0.0000 0.0010 0.0004
MXAutogradBackwardEx 678 375.0660 0.5250 0.9450 0.5532
MXNDArrayGetContext 10532 4.5980 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22298 573.0180 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2712 6644.3608 0.7010 3.7500 2.4500
FullyConnected 4548 439.6460 0.0080 0.3430 0.0967
argmax 1516 30.1080 0.0050 0.0480 0.0199
_backward_pick 1356 21.4730 0.0130 0.0320 0.0158
_backward_mul_scalar 1356 5.9240 0.0030 0.0220 0.0044
DeleteVariable 57120 401.9090 0.0010 0.9010 0.0070
Flatten 1516 62.7990 0.0080 0.1120 0.0414
Activation 6064 281.9510 0.0030 1.9440 0.0465
SetValueOp 1356 8.7730 0.0050 0.0180 0.0065
pick 1356 10.6750 0.0060 0.0280 0.0079
Pooling 3032 4252.6040 0.0640 3.1720 1.4026
Convolution 3032 17302.4023 0.2260 13.2050 5.7066
CopyCPU2CPU 1516 8.5770 0.0030 0.0360 0.0057
log_softmax 1356 42.2160 0.0150 0.0550 0.0311
_backward_copy 1356 70.0140 0.0220 0.0760 0.0516
_backward_Convolution 2712 25115.9844 2.6370 12.5610 9.2611
_mul_scalar 1356 4.8580 0.0020 0.0230 0.0036
_backward_log_softmax 1356 23.1050 0.0090 0.0400 0.0170
mean 4228 81.1780 0.0080 0.0730 0.0192
_backward_mean 1356 14.8800 0.0080 0.1030 0.0110
_backward_FullyConnected 4068 780.4140 0.0280 0.5940 0.1918
WaitForVar 2872 25.6320 0.0060 0.0280 0.0089
broadcast_equal 1516 7.5730 0.0030 0.0110 0.0050
sgd_update 13560 134.2430 0.0020 0.0790 0.0099
_backward_Activation 5424 400.9020 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52646 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4554 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 678 2659.5010 0.3060 73.5850 3.9226
MXNDArrayCreateEx 759 26.1420 0.0310 0.0640 0.0344
MXNDArrayWaitAll 679 14750.1396 5.6130 25.0610 21.7233
MXNDArraySyncCopyToCPU 1438 11273.4639 0.1180 24.1980 7.8397
MXNDArrayGetDType 1438 0.5240 0.0000 0.0010 0.0004
MXNDArrayGetShape 3635 1.7010 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1518 13.2970 0.0070 0.0280 0.0088
MXNDArrayFree 17057 174.0310 0.0000 0.9140 0.0102
MXNet C API Calls 77053 564.4100 487.3580 564.4100 38.5260
MXNet C API Concurrency 154105 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1358 0.6420 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6790 3.5330 0.0000 0.0230 0.0005
MXNDArraySetGradState 6790 3.4650 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1358 0.5780 0.0000 0.0010 0.0004
MXAutogradBackwardEx 679 375.6220 0.5250 0.9450 0.5532
MXNDArrayGetContext 10546 4.6020 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22329 573.8210 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2716 6654.0391 0.7010 3.7500 2.4499
FullyConnected 4554 440.2690 0.0080 0.3430 0.0967
argmax 1518 30.1510 0.0050 0.0480 0.0199
_backward_pick 1358 21.5040 0.0130 0.0320 0.0158
_backward_mul_scalar 1358 5.9330 0.0030 0.0220 0.0044
DeleteVariable 57200 402.4640 0.0010 0.9010 0.0070
Flatten 1518 62.8800 0.0080 0.1120 0.0414
Activation 6072 282.3520 0.0030 1.9440 0.0465
SetValueOp 1358 8.7860 0.0050 0.0180 0.0065
pick 1358 10.6890 0.0060 0.0280 0.0079
Pooling 3036 4257.9961 0.0640 3.1720 1.4025
Convolution 3036 17324.5781 0.2260 13.2050 5.7064
CopyCPU2CPU 1518 8.5880 0.0030 0.0360 0.0057
log_softmax 1358 42.2760 0.0150 0.0550 0.0311
_backward_copy 1358 70.1150 0.0220 0.0760 0.0516
_backward_Convolution 2716 25152.7598 2.6370 12.5610 9.2610
_mul_scalar 1358 4.8650 0.0020 0.0230 0.0036
_backward_log_softmax 1358 23.1480 0.0090 0.0400 0.0170
mean 4234 81.2840 0.0080 0.0730 0.0192
_backward_mean 1358 14.9020 0.0080 0.1030 0.0110
_backward_FullyConnected 4074 781.5880 0.0280 0.5940 0.1918
WaitForVar 2876 25.6670 0.0060 0.0280 0.0089
broadcast_equal 1518 7.5820 0.0030 0.0110 0.0050
sgd_update 13580 134.4530 0.0020 0.0790 0.0099
_backward_Activation 5432 401.4630 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52720 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4560 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 679 2663.4099 0.3060 73.5850 3.9225
MXNDArrayCreateEx 760 26.1770 0.0310 0.0640 0.0344
MXNDArrayWaitAll 680 14771.8896 5.6130 25.0610 21.7234
MXNDArraySyncCopyToCPU 1440 11287.9648 0.1180 24.1980 7.8389
MXNDArrayGetDType 1440 0.5240 0.0000 0.0010 0.0004
MXNDArrayGetShape 3640 1.7060 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1520 13.3140 0.0070 0.0280 0.0088
MXNDArrayFree 17080 174.2610 0.0000 0.9140 0.0102
MXNet C API Calls 77160 564.5170 487.3580 564.5170 38.5795
MXNet C API Concurrency 154319 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1360 0.6440 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6800 3.5360 0.0000 0.0230 0.0005
MXNDArraySetGradState 6800 3.4690 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1360 0.5780 0.0000 0.0010 0.0004
MXAutogradBackwardEx 680 376.1570 0.5250 0.9450 0.5532
MXNDArrayGetContext 10560 4.6100 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22360 574.6120 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2720 6663.8271 0.7010 3.7500 2.4499
FullyConnected 4560 440.8810 0.0080 0.3430 0.0967
argmax 1520 30.1930 0.0050 0.0480 0.0199
_backward_pick 1360 21.5350 0.0130 0.0320 0.0158
_backward_mul_scalar 1360 5.9410 0.0030 0.0220 0.0044
DeleteVariable 57280 403.0100 0.0010 0.9010 0.0070
Flatten 1520 62.9600 0.0080 0.1120 0.0414
Activation 6080 282.7390 0.0030 1.9440 0.0465
SetValueOp 1360 8.7990 0.0050 0.0180 0.0065
pick 1360 10.7070 0.0060 0.0280 0.0079
Pooling 3040 4263.3550 0.0640 3.1720 1.4024
Convolution 3040 17346.6699 0.2260 13.2050 5.7061
CopyCPU2CPU 1520 8.5980 0.0030 0.0360 0.0057
log_softmax 1360 42.3360 0.0150 0.0550 0.0311
_backward_copy 1360 70.2530 0.0220 0.0760 0.0517
_backward_Convolution 2720 25189.5430 2.6370 12.5610 9.2609
_mul_scalar 1360 4.8720 0.0020 0.0230 0.0036
_backward_log_softmax 1360 23.1900 0.0090 0.0400 0.0171
mean 4240 81.3900 0.0080 0.0730 0.0192
_backward_mean 1360 14.9260 0.0080 0.1030 0.0110
_backward_FullyConnected 4080 782.7730 0.0280 0.5940 0.1919
WaitForVar 2880 25.7020 0.0060 0.0280 0.0089
broadcast_equal 1520 7.5920 0.0030 0.0110 0.0050
sgd_update 13600 134.6600 0.0020 0.0790 0.0099
_backward_Activation 5440 402.0580 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52794 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4566 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 680 2667.3169 0.3060 73.5850 3.9225
MXNDArrayCreateEx 761 26.2120 0.0310 0.0640 0.0344
MXNDArrayWaitAll 681 14793.7637 5.6130 25.0610 21.7236
MXNDArraySyncCopyToCPU 1442 11302.3477 0.1180 24.1980 7.8380
MXNDArrayGetDType 1442 0.5240 0.0000 0.0010 0.0004
MXNDArrayGetShape 3645 1.7090 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1522 13.3310 0.0070 0.0280 0.0088
MXNDArrayFree 17103 174.4910 0.0000 0.9140 0.0102
MXNet C API Calls 77267 564.6240 487.3580 564.6240 38.6330
MXNet C API Concurrency 154533 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1362 0.6440 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6810 3.5390 0.0000 0.0230 0.0005
MXNDArraySetGradState 6810 3.4780 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1362 0.5790 0.0000 0.0010 0.0004
MXAutogradBackwardEx 681 376.7150 0.5250 0.9450 0.5532
MXNDArrayGetContext 10574 4.6170 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22391 575.3780 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2724 6673.5488 0.7010 3.7500 2.4499
FullyConnected 4566 441.4050 0.0080 0.3430 0.0967
argmax 1522 30.2310 0.0050 0.0480 0.0199
_backward_pick 1362 21.5680 0.0130 0.0320 0.0158
_backward_mul_scalar 1362 5.9500 0.0030 0.0220 0.0044
DeleteVariable 57360 403.5550 0.0010 0.9010 0.0070
Flatten 1522 63.0430 0.0080 0.1120 0.0414
Activation 6088 283.0910 0.0030 1.9440 0.0465
SetValueOp 1362 8.8120 0.0050 0.0180 0.0065
pick 1362 10.7220 0.0060 0.0280 0.0079
Pooling 3044 4268.7300 0.0640 3.1720 1.4023
Convolution 3044 17368.8457 0.2260 13.2050 5.7059
CopyCPU2CPU 1522 8.6080 0.0030 0.0360 0.0057
log_softmax 1362 42.3960 0.0150 0.0550 0.0311
_backward_copy 1362 70.3560 0.0220 0.0760 0.0517
_backward_Convolution 2724 25226.6289 2.6370 12.5610 9.2609
_mul_scalar 1362 4.8790 0.0020 0.0230 0.0036
_backward_log_softmax 1362 23.2230 0.0090 0.0400 0.0171
mean 4246 81.5120 0.0080 0.0730 0.0192
_backward_mean 1362 14.9480 0.0080 0.1030 0.0110
_backward_FullyConnected 4086 783.8410 0.0280 0.5940 0.1918
WaitForVar 2884 25.7360 0.0060 0.0280 0.0089
broadcast_equal 1522 7.6010 0.0030 0.0110 0.0050
sgd_update 13620 134.8620 0.0020 0.0790 0.0099
_backward_Activation 5448 402.5990 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52868 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4572 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 681 2671.2429 0.3060 73.5850 3.9225
MXNDArrayCreateEx 762 26.2470 0.0310 0.0640 0.0344
MXNDArrayWaitAll 682 14815.5742 5.6130 25.0610 21.7237
MXNDArraySyncCopyToCPU 1444 11316.8174 0.1180 24.1980 7.8371
MXNDArrayGetDType 1444 0.5240 0.0000 0.0010 0.0004
MXNDArrayGetShape 3650 1.7100 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1524 13.3490 0.0070 0.0280 0.0088
MXNDArrayFree 17126 174.7240 0.0000 0.9140 0.0102
MXNet C API Calls 77374 564.7310 487.3580 564.7310 38.6865
MXNet C API Concurrency 154747 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1364 0.6440 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6820 3.5430 0.0000 0.0230 0.0005
MXNDArraySetGradState 6820 3.4850 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1364 0.5790 0.0000 0.0010 0.0004
MXAutogradBackwardEx 682 377.2600 0.5250 0.9450 0.5532
MXNDArrayGetContext 10588 4.6200 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22422 576.1600 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2728 6683.2168 0.7010 3.7500 2.4499
FullyConnected 4572 441.9630 0.0080 0.3430 0.0967
argmax 1524 30.2690 0.0050 0.0480 0.0199
_backward_pick 1364 21.6000 0.0130 0.0320 0.0158
_backward_mul_scalar 1364 5.9590 0.0030 0.0220 0.0044
DeleteVariable 57440 404.0950 0.0010 0.9010 0.0070
Flatten 1524 63.1330 0.0080 0.1120 0.0414
Activation 6096 283.4610 0.0030 1.9440 0.0465
SetValueOp 1364 8.8250 0.0050 0.0180 0.0065
pick 1364 10.7380 0.0060 0.0280 0.0079
Pooling 3048 4274.1650 0.0640 3.1720 1.4023
Convolution 3048 17391.1035 0.2260 13.2050 5.7057
CopyCPU2CPU 1524 8.6180 0.0030 0.0360 0.0057
log_softmax 1364 42.4570 0.0150 0.0550 0.0311
_backward_copy 1364 70.4580 0.0220 0.0760 0.0517
_backward_Convolution 2728 25263.6406 2.6370 12.5610 9.2609
_mul_scalar 1364 4.8860 0.0020 0.0230 0.0036
_backward_log_softmax 1364 23.2550 0.0090 0.0400 0.0170
mean 4252 81.6290 0.0080 0.0730 0.0192
_backward_mean 1364 14.9700 0.0080 0.1030 0.0110
_backward_FullyConnected 4092 784.9160 0.0280 0.5940 0.1918
WaitForVar 2888 25.7720 0.0060 0.0280 0.0089
broadcast_equal 1524 7.6110 0.0030 0.0110 0.0050
sgd_update 13640 135.0650 0.0020 0.0790 0.0099
_backward_Activation 5456 403.1580 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 52942 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4578 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 682 2675.1699 0.3060 73.5850 3.9225
MXNDArrayCreateEx 763 26.2820 0.0310 0.0640 0.0344
MXNDArrayWaitAll 683 14837.3232 5.6130 25.0610 21.7238
MXNDArraySyncCopyToCPU 1446 11331.2607 0.1180 24.1980 7.8363
MXNDArrayGetDType 1446 0.5240 0.0000 0.0010 0.0004
MXNDArrayGetShape 3655 1.7110 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1526 13.3660 0.0070 0.0280 0.0088
MXNDArrayFree 17149 174.9630 0.0000 0.9140 0.0102
MXNet C API Calls 77481 564.8380 487.3580 564.8380 38.7400
MXNet C API Concurrency 154961 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1366 0.6460 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6830 3.5490 0.0000 0.0230 0.0005
MXNDArraySetGradState 6830 3.4870 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1366 0.5800 0.0000 0.0010 0.0004
MXAutogradBackwardEx 683 377.8360 0.5250 0.9450 0.5532
MXNDArrayGetContext 10602 4.6280 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22453 576.9540 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2732 6692.9131 0.7010 3.7500 2.4498
FullyConnected 4578 442.5650 0.0080 0.3430 0.0967
argmax 1526 30.3070 0.0050 0.0480 0.0199
_backward_pick 1366 21.6330 0.0130 0.0320 0.0158
_backward_mul_scalar 1366 5.9680 0.0030 0.0220 0.0044
DeleteVariable 57520 404.6790 0.0010 0.9010 0.0070
Flatten 1526 63.2270 0.0080 0.1120 0.0414
Activation 6104 283.8330 0.0030 1.9440 0.0465
SetValueOp 1366 8.8380 0.0050 0.0180 0.0065
pick 1366 10.7550 0.0060 0.0280 0.0079
Pooling 3052 4279.5361 0.0640 3.1720 1.4022
Convolution 3052 17413.3691 0.2260 13.2050 5.7056
CopyCPU2CPU 1526 8.6290 0.0030 0.0360 0.0057
log_softmax 1366 42.5310 0.0150 0.0550 0.0311
_backward_copy 1366 70.5600 0.0220 0.0760 0.0517
_backward_Convolution 2732 25300.5137 2.6370 12.5610 9.2608
_mul_scalar 1366 4.8920 0.0020 0.0230 0.0036
_backward_log_softmax 1366 23.2870 0.0090 0.0400 0.0170
mean 4258 81.7370 0.0080 0.0730 0.0192
_backward_mean 1366 14.9920 0.0080 0.1030 0.0110
_backward_FullyConnected 4098 786.1270 0.0280 0.5940 0.1918
WaitForVar 2892 25.8060 0.0060 0.0280 0.0089
broadcast_equal 1526 7.6210 0.0030 0.0110 0.0050
sgd_update 13660 135.3080 0.0020 0.0790 0.0099
_backward_Activation 5464 403.7560 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53016 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4584 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 683 2679.1260 0.3060 73.5850 3.9226
MXNDArrayCreateEx 764 26.3180 0.0310 0.0640 0.0344
MXNDArrayWaitAll 684 14859.0156 5.6130 25.0610 21.7237
MXNDArraySyncCopyToCPU 1448 11345.7930 0.1180 24.1980 7.8355
MXNDArrayGetDType 1448 0.5250 0.0000 0.0010 0.0004
MXNDArrayGetShape 3660 1.7130 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1528 13.3830 0.0070 0.0280 0.0088
MXNDArrayFree 17172 175.2260 0.0000 0.9140 0.0102
MXNet C API Calls 77588 564.9450 487.3580 564.9450 38.7935
MXNet C API Concurrency 155175 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1368 0.6480 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6840 3.5550 0.0000 0.0230 0.0005
MXNDArraySetGradState 6840 3.4910 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1368 0.5810 0.0000 0.0010 0.0004
MXAutogradBackwardEx 684 378.3890 0.5250 0.9450 0.5532
MXNDArrayGetContext 10616 4.6340 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22484 577.7440 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2736 6702.6099 0.7010 3.7500 2.4498
FullyConnected 4584 443.1690 0.0080 0.3430 0.0967
argmax 1528 30.3450 0.0050 0.0480 0.0199
_backward_pick 1368 21.6690 0.0130 0.0320 0.0158
_backward_mul_scalar 1368 5.9760 0.0030 0.0220 0.0044
DeleteVariable 57600 405.2740 0.0010 0.9010 0.0070
Flatten 1528 63.3280 0.0080 0.1120 0.0414
Activation 6112 284.2170 0.0030 1.9440 0.0465
SetValueOp 1368 8.8520 0.0050 0.0180 0.0065
pick 1368 10.7710 0.0060 0.0280 0.0079
Pooling 3056 4284.9219 0.0640 3.1720 1.4021
Convolution 3056 17435.5273 0.2260 13.2050 5.7053
CopyCPU2CPU 1528 8.6400 0.0030 0.0360 0.0057
log_softmax 1368 42.6060 0.0150 0.0550 0.0311
_backward_copy 1368 70.6630 0.0220 0.0760 0.0517
_backward_Convolution 2736 25337.2559 2.6370 12.5610 9.2607
_mul_scalar 1368 4.8990 0.0020 0.0230 0.0036
_backward_log_softmax 1368 23.3200 0.0090 0.0400 0.0170
mean 4264 81.8490 0.0080 0.0730 0.0192
_backward_mean 1368 15.0150 0.0080 0.1030 0.0110
_backward_FullyConnected 4104 787.3310 0.0280 0.5940 0.1918
WaitForVar 2896 25.8400 0.0060 0.0280 0.0089
broadcast_equal 1528 7.6320 0.0030 0.0110 0.0050
sgd_update 13680 135.5160 0.0020 0.0790 0.0099
_backward_Activation 5472 404.3670 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53090 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4590 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 684 2683.1069 0.3060 73.5850 3.9227
MXNDArrayCreateEx 765 26.3570 0.0310 0.0640 0.0345
MXNDArrayWaitAll 685 14880.8887 5.6130 25.0610 21.7239
MXNDArraySyncCopyToCPU 1450 11360.2393 0.1180 24.1980 7.8346
MXNDArrayGetDType 1450 0.5260 0.0000 0.0010 0.0004
MXNDArrayGetShape 3665 1.7140 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1530 13.4020 0.0070 0.0280 0.0088
MXNDArrayFree 17195 175.4780 0.0000 0.9140 0.0102
MXNet C API Calls 77695 565.0520 487.3580 565.0520 38.8470
MXNet C API Concurrency 155389 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1370 0.6480 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6850 3.5600 0.0000 0.0230 0.0005
MXNDArraySetGradState 6850 3.4960 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1370 0.5810 0.0000 0.0010 0.0004
MXAutogradBackwardEx 685 378.9710 0.5250 0.9450 0.5532
MXNDArrayGetContext 10630 4.6390 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22515 578.5540 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2740 6712.2681 0.7010 3.7500 2.4497
FullyConnected 4590 443.7290 0.0080 0.3430 0.0967
argmax 1530 30.3850 0.0050 0.0480 0.0199
_backward_pick 1370 21.7040 0.0130 0.0320 0.0158
_backward_mul_scalar 1370 5.9860 0.0030 0.0220 0.0044
DeleteVariable 57680 405.8560 0.0010 0.9010 0.0070
Flatten 1530 63.4430 0.0080 0.1120 0.0415
Activation 6120 284.5980 0.0030 1.9440 0.0465
SetValueOp 1370 8.8660 0.0050 0.0180 0.0065
pick 1370 10.7880 0.0060 0.0280 0.0079
Pooling 3060 4290.3540 0.0640 3.1720 1.4021
Convolution 3060 17457.8438 0.2260 13.2050 5.7052
CopyCPU2CPU 1530 8.6510 0.0030 0.0360 0.0057
log_softmax 1370 42.6680 0.0150 0.0550 0.0311
_backward_copy 1370 70.7680 0.0220 0.0760 0.0517
_backward_Convolution 2740 25374.3516 2.6370 12.5610 9.2607
_mul_scalar 1370 4.9060 0.0020 0.0230 0.0036
_backward_log_softmax 1370 23.3530 0.0090 0.0400 0.0170
mean 4270 81.9730 0.0080 0.0730 0.0192
_backward_mean 1370 15.0410 0.0080 0.1030 0.0110
_backward_FullyConnected 4110 788.4220 0.0280 0.5940 0.1918
WaitForVar 2900 25.8770 0.0060 0.0280 0.0089
broadcast_equal 1530 7.6430 0.0030 0.0110 0.0050
sgd_update 13700 135.7220 0.0020 0.0790 0.0099
_backward_Activation 5480 404.9550 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53164 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4596 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 685 2687.0730 0.3060 73.5850 3.9227
MXNDArrayCreateEx 766 26.3930 0.0310 0.0640 0.0345
MXNDArrayWaitAll 686 14892.6445 5.6130 25.0610 21.7094
MXNDArraySyncCopyToCPU 1452 11384.4072 0.1180 24.1980 7.8405
MXNDArrayGetDType 1452 0.5260 0.0000 0.0010 0.0004
MXNDArrayGetShape 3670 1.7170 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1532 13.4190 0.0070 0.0280 0.0088
MXNDArrayFree 17218 175.7190 0.0000 0.9140 0.0102
MXNet C API Calls 77802 565.1590 487.3580 565.1590 38.9005
MXNet C API Concurrency 155603 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1372 0.6480 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6860 3.5640 0.0000 0.0230 0.0005
MXNDArraySetGradState 6860 3.5010 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1372 0.5810 0.0000 0.0010 0.0004
MXAutogradBackwardEx 686 379.5290 0.5250 0.9450 0.5532
MXNDArrayGetContext 10644 4.6470 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22546 579.3620 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2744 6721.9980 0.7010 3.7500 2.4497
FullyConnected 4596 444.2840 0.0080 0.3430 0.0967
argmax 1532 30.4250 0.0050 0.0480 0.0199
_backward_pick 1372 21.7350 0.0130 0.0320 0.0158
_backward_mul_scalar 1372 5.9960 0.0030 0.0220 0.0044
DeleteVariable 57760 406.4270 0.0010 0.9010 0.0070
Flatten 1532 63.5280 0.0080 0.1120 0.0415
Activation 6128 284.9550 0.0030 1.9440 0.0465
SetValueOp 1372 8.8790 0.0050 0.0180 0.0065
pick 1372 10.8040 0.0060 0.0280 0.0079
Pooling 3064 4295.7031 0.0640 3.1720 1.4020
Convolution 3064 17480.2207 0.2260 13.2050 5.7050
CopyCPU2CPU 1532 8.7190 0.0030 0.0360 0.0057
log_softmax 1372 42.7290 0.0150 0.0550 0.0311
_backward_copy 1372 70.8740 0.0220 0.0760 0.0517
_backward_Convolution 2744 25411.4062 2.6370 12.5610 9.2607
_mul_scalar 1372 4.9130 0.0020 0.0230 0.0036
_backward_log_softmax 1372 23.3860 0.0090 0.0400 0.0170
mean 4276 82.0490 0.0080 0.0730 0.0192
_backward_mean 1372 15.0640 0.0080 0.1030 0.0110
_backward_FullyConnected 4116 789.4970 0.0280 0.5940 0.1918
WaitForVar 2904 25.9090 0.0060 0.0280 0.0089
broadcast_equal 1532 7.6540 0.0030 0.0110 0.0050
sgd_update 13720 135.9240 0.0020 0.0790 0.0099
_backward_Activation 5488 405.5120 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53238 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4602 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 686 2691.0100 0.3060 73.5850 3.9228
MXNDArrayCreateEx 767 26.4270 0.0310 0.0640 0.0345
MXNDArrayWaitAll 687 14914.4180 5.6130 25.0610 21.7095
MXNDArraySyncCopyToCPU 1454 11398.8721 0.1180 24.1980 7.8397
MXNDArrayGetDType 1454 0.5260 0.0000 0.0010 0.0004
MXNDArrayGetShape 3675 1.7180 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1534 13.4360 0.0070 0.0280 0.0088
MXNDArrayFree 17241 175.9510 0.0000 0.9140 0.0102
MXNet C API Calls 77909 565.2660 487.3580 565.2660 38.9540
MXNet C API Concurrency 155817 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1374 0.6500 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6870 3.5680 0.0000 0.0230 0.0005
MXNDArraySetGradState 6870 3.5030 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1374 0.5820 0.0000 0.0010 0.0004
MXAutogradBackwardEx 687 380.0800 0.5250 0.9450 0.5532
MXNDArrayGetContext 10658 4.6570 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22577 580.1500 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2748 6731.6782 0.7010 3.7500 2.4497
FullyConnected 4602 444.8770 0.0080 0.3430 0.0967
argmax 1534 30.4720 0.0050 0.0480 0.0199
_backward_pick 1374 21.7700 0.0130 0.0320 0.0158
_backward_mul_scalar 1374 6.0040 0.0030 0.0220 0.0044
DeleteVariable 57840 406.9830 0.0010 0.9010 0.0070
Flatten 1534 63.6170 0.0080 0.1120 0.0415
Activation 6136 285.3210 0.0030 1.9440 0.0465
SetValueOp 1374 8.8920 0.0050 0.0180 0.0065
pick 1374 10.8210 0.0060 0.0280 0.0079
Pooling 3068 4301.0801 0.0640 3.1720 1.4019
Convolution 3068 17502.4395 0.2260 13.2050 5.7048
CopyCPU2CPU 1534 8.7310 0.0030 0.0360 0.0057
log_softmax 1374 42.8000 0.0150 0.0550 0.0311
_backward_copy 1374 70.9770 0.0220 0.0760 0.0517
_backward_Convolution 2748 25448.3145 2.6370 12.5610 9.2607
_mul_scalar 1374 4.9200 0.0020 0.0230 0.0036
_backward_log_softmax 1374 23.4180 0.0090 0.0400 0.0170
mean 4282 82.1580 0.0080 0.0730 0.0192
_backward_mean 1374 15.0860 0.0080 0.1030 0.0110
_backward_FullyConnected 4122 790.6880 0.0280 0.5940 0.1918
WaitForVar 2908 25.9450 0.0060 0.0280 0.0089
broadcast_equal 1534 7.6640 0.0030 0.0110 0.0050
sgd_update 13740 136.1530 0.0020 0.0790 0.0099
_backward_Activation 5496 406.0730 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53312 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4608 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 687 2694.9419 0.3060 73.5850 3.9228
MXNDArrayCreateEx 768 26.4620 0.0310 0.0640 0.0345
MXNDArrayWaitAll 688 14936.1572 5.6130 25.0610 21.7095
MXNDArraySyncCopyToCPU 1456 11413.3467 0.1180 24.1980 7.8388
MXNDArrayGetDType 1456 0.5260 0.0000 0.0010 0.0004
MXNDArrayGetShape 3680 1.7210 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1536 13.4540 0.0070 0.0280 0.0088
MXNDArrayFree 17264 176.1850 0.0000 0.9140 0.0102
MXNet C API Calls 78016 565.3730 487.3580 565.3730 39.0075
MXNet C API Concurrency 156031 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1376 0.6510 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6880 3.5750 0.0000 0.0230 0.0005
MXNDArraySetGradState 6880 3.5070 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1376 0.5830 0.0000 0.0010 0.0004
MXAutogradBackwardEx 688 380.6310 0.5250 0.9450 0.5532
MXNDArrayGetContext 10672 4.6630 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22608 580.9510 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2752 6741.4849 0.7010 3.7500 2.4497
FullyConnected 4608 445.4750 0.0080 0.3430 0.0967
argmax 1536 30.5160 0.0050 0.0480 0.0199
_backward_pick 1376 21.8050 0.0130 0.0320 0.0158
_backward_mul_scalar 1376 6.0120 0.0030 0.0220 0.0044
DeleteVariable 57920 407.5530 0.0010 0.9010 0.0070
Flatten 1536 63.7060 0.0080 0.1120 0.0415
Activation 6144 285.7060 0.0030 1.9440 0.0465
SetValueOp 1376 8.9050 0.0050 0.0180 0.0065
pick 1376 10.8390 0.0060 0.0280 0.0079
Pooling 3072 4306.4888 0.0640 3.1720 1.4019
Convolution 3072 17524.5586 0.2260 13.2050 5.7046
CopyCPU2CPU 1536 8.7420 0.0030 0.0360 0.0057
log_softmax 1376 42.8630 0.0150 0.0550 0.0312
_backward_copy 1376 71.0790 0.0220 0.0760 0.0517
_backward_Convolution 2752 25485.0801 2.6370 12.5610 9.2606
_mul_scalar 1376 4.9260 0.0020 0.0230 0.0036
_backward_log_softmax 1376 23.4590 0.0090 0.0400 0.0170
mean 4288 82.2650 0.0080 0.0730 0.0192
_backward_mean 1376 15.1090 0.0080 0.1030 0.0110
_backward_FullyConnected 4128 791.8800 0.0280 0.5940 0.1918
WaitForVar 2912 25.9810 0.0060 0.0280 0.0089
broadcast_equal 1536 7.6750 0.0030 0.0110 0.0050
sgd_update 13760 136.3670 0.0020 0.0790 0.0099
_backward_Activation 5504 406.6400 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53386 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4614 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 688 2698.8579 0.3060 73.5850 3.9228
MXNDArrayCreateEx 769 26.4960 0.0310 0.0640 0.0345
MXNDArrayWaitAll 689 14958.0195 5.6130 25.0610 21.7098
MXNDArraySyncCopyToCPU 1458 11427.7852 0.1180 24.1980 7.8380
MXNDArrayGetDType 1458 0.5270 0.0000 0.0010 0.0004
MXNDArrayGetShape 3685 1.7210 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1538 13.4710 0.0070 0.0280 0.0088
MXNDArrayFree 17287 176.4250 0.0000 0.9140 0.0102
MXNet C API Calls 78123 565.4800 487.3580 565.4800 39.0610
MXNet C API Concurrency 156245 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1378 0.6520 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6890 3.5810 0.0000 0.0230 0.0005
MXNDArraySetGradState 6890 3.5130 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1378 0.5850 0.0000 0.0010 0.0004
MXAutogradBackwardEx 689 381.1700 0.5250 0.9450 0.5532
MXNDArrayGetContext 10686 4.6710 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22639 581.7280 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2756 6751.1812 0.7010 3.7500 2.4496
FullyConnected 4614 446.0000 0.0080 0.3430 0.0967
argmax 1538 30.5560 0.0050 0.0480 0.0199
_backward_pick 1378 21.8380 0.0130 0.0320 0.0158
_backward_mul_scalar 1378 6.0210 0.0030 0.0220 0.0044
DeleteVariable 58000 408.0880 0.0010 0.9010 0.0070
Flatten 1538 63.7890 0.0080 0.1120 0.0415
Activation 6152 286.0620 0.0030 1.9440 0.0465
SetValueOp 1378 8.9190 0.0050 0.0180 0.0065
pick 1378 10.8550 0.0060 0.0280 0.0079
Pooling 3076 4311.8418 0.0640 3.1720 1.4018
Convolution 3076 17546.8145 0.2260 13.2050 5.7044
CopyCPU2CPU 1538 8.7530 0.0030 0.0360 0.0057
log_softmax 1378 42.9230 0.0150 0.0550 0.0311
_backward_copy 1378 71.1820 0.0220 0.0760 0.0517
_backward_Convolution 2756 25522.1738 2.6370 12.5610 9.2606
_mul_scalar 1378 4.9330 0.0020 0.0230 0.0036
_backward_log_softmax 1378 23.4910 0.0090 0.0400 0.0170
mean 4294 82.3850 0.0080 0.0730 0.0192
_backward_mean 1378 15.1320 0.0080 0.1030 0.0110
_backward_FullyConnected 4134 793.0030 0.0280 0.5940 0.1918
WaitForVar 2916 26.0160 0.0060 0.0280 0.0089
broadcast_equal 1538 7.6860 0.0030 0.0110 0.0050
sgd_update 13780 136.5710 0.0020 0.0790 0.0099
_backward_Activation 5512 407.2000 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53460 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4620 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 689 2702.7700 0.3060 73.5850 3.9227
MXNDArrayCreateEx 770 26.5300 0.0310 0.0640 0.0345
MXNDArrayWaitAll 690 14979.8799 5.6130 25.0610 21.7100
MXNDArraySyncCopyToCPU 1460 11442.2842 0.1180 24.1980 7.8372
MXNDArrayGetDType 1460 0.5280 0.0000 0.0010 0.0004
MXNDArrayGetShape 3690 1.7230 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1540 13.4880 0.0070 0.0280 0.0088
MXNDArrayFree 17310 176.6600 0.0000 0.9140 0.0102
MXNet C API Calls 78230 565.5870 487.3580 565.5870 39.1145
MXNet C API Concurrency 156459 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1380 0.6540 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6900 3.5870 0.0000 0.0230 0.0005
MXNDArraySetGradState 6900 3.5150 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1380 0.5860 0.0000 0.0010 0.0004
MXAutogradBackwardEx 690 381.7110 0.5250 0.9450 0.5532
MXNDArrayGetContext 10700 4.6750 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22670 582.5300 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2760 6760.9771 0.7010 3.7500 2.4496
FullyConnected 4620 446.5260 0.0080 0.3430 0.0967
argmax 1540 30.5950 0.0050 0.0480 0.0199
_backward_pick 1380 21.8690 0.0130 0.0320 0.0158
_backward_mul_scalar 1380 6.0300 0.0030 0.0220 0.0044
DeleteVariable 58080 408.6280 0.0010 0.9010 0.0070
Flatten 1540 63.8700 0.0080 0.1120 0.0415
Activation 6160 286.4150 0.0030 1.9440 0.0465
SetValueOp 1380 8.9320 0.0050 0.0180 0.0065
pick 1380 10.8700 0.0060 0.0280 0.0079
Pooling 3080 4317.2178 0.0640 3.1720 1.4017
Convolution 3080 17569.1230 0.2260 13.2050 5.7043
CopyCPU2CPU 1540 8.7630 0.0030 0.0360 0.0057
log_softmax 1380 42.9820 0.0150 0.0550 0.0311
_backward_copy 1380 71.2860 0.0220 0.0760 0.0517
_backward_Convolution 2760 25559.2246 2.6370 12.5610 9.2606
_mul_scalar 1380 4.9400 0.0020 0.0230 0.0036
_backward_log_softmax 1380 23.5230 0.0090 0.0400 0.0170
mean 4300 82.5000 0.0080 0.0730 0.0192
_backward_mean 1380 15.1540 0.0080 0.1030 0.0110
_backward_FullyConnected 4140 794.0890 0.0280 0.5940 0.1918
WaitForVar 2920 26.0520 0.0060 0.0280 0.0089
broadcast_equal 1540 7.6960 0.0030 0.0110 0.0050
sgd_update 13800 136.7740 0.0020 0.0790 0.0099
_backward_Activation 5520 407.7690 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53534 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4626 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 690 2706.6851 0.3060 73.5850 3.9227
MXNDArrayCreateEx 771 26.5640 0.0310 0.0640 0.0345
MXNDArrayWaitAll 691 15001.6787 5.6130 25.0610 21.7101
MXNDArraySyncCopyToCPU 1462 11456.8623 0.1180 24.1980 7.8364
MXNDArrayGetDType 1462 0.5280 0.0000 0.0010 0.0004
MXNDArrayGetShape 3695 1.7260 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1542 13.5060 0.0070 0.0280 0.0088
MXNDArrayFree 17333 176.8930 0.0000 0.9140 0.0102
MXNet C API Calls 78337 565.6940 487.3580 565.6940 39.1680
MXNet C API Concurrency 156673 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1382 0.6550 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6910 3.5940 0.0000 0.0230 0.0005
MXNDArraySetGradState 6910 3.5220 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1382 0.5870 0.0000 0.0010 0.0004
MXAutogradBackwardEx 691 382.2480 0.5250 0.9450 0.5532
MXNDArrayGetContext 10714 4.6800 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22701 583.3020 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2764 6770.7739 0.7010 3.7500 2.4496
FullyConnected 4626 447.1300 0.0080 0.3430 0.0967
argmax 1542 30.6390 0.0050 0.0480 0.0199
_backward_pick 1382 21.9020 0.0130 0.0320 0.0158
_backward_mul_scalar 1382 6.0380 0.0030 0.0220 0.0044
DeleteVariable 58160 409.1520 0.0010 0.9010 0.0070
Flatten 1542 63.9530 0.0080 0.1120 0.0415
Activation 6168 286.7870 0.0030 1.9440 0.0465
SetValueOp 1382 8.9460 0.0050 0.0180 0.0065
pick 1382 10.8870 0.0060 0.0280 0.0079
Pooling 3084 4322.5869 0.0640 3.1720 1.4016
Convolution 3084 17591.3359 0.2260 13.2050 5.7041
CopyCPU2CPU 1542 8.7730 0.0030 0.0360 0.0057
log_softmax 1382 43.0540 0.0150 0.0550 0.0312
_backward_copy 1382 71.3870 0.0220 0.0760 0.0517
_backward_Convolution 2764 25596.1074 2.6370 12.5610 9.2605
_mul_scalar 1382 4.9470 0.0020 0.0230 0.0036
_backward_log_softmax 1382 23.5650 0.0090 0.0400 0.0171
mean 4306 82.6070 0.0080 0.0730 0.0192
_backward_mean 1382 15.1760 0.0080 0.1030 0.0110
_backward_FullyConnected 4146 795.3090 0.0280 0.5940 0.1918
WaitForVar 2924 26.0870 0.0060 0.0280 0.0089
broadcast_equal 1542 7.7070 0.0030 0.0110 0.0050
sgd_update 13820 136.9840 0.0020 0.0790 0.0099
_backward_Activation 5528 408.3440 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53608 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4632 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 691 2710.6121 0.3060 73.5850 3.9227
MXNDArrayCreateEx 772 26.5980 0.0310 0.0640 0.0345
MXNDArrayWaitAll 692 15023.3750 5.6130 25.0610 21.7101
MXNDArraySyncCopyToCPU 1464 11471.3838 0.1180 24.1980 7.8356
MXNDArrayGetDType 1464 0.5280 0.0000 0.0010 0.0004
MXNDArrayGetShape 3700 1.7280 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1544 13.5230 0.0070 0.0280 0.0088
MXNDArrayFree 17356 177.1260 0.0000 0.9140 0.0102
MXNet C API Calls 78444 565.8010 487.3580 565.8010 39.2215
MXNet C API Concurrency 156887 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1384 0.6560 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6920 3.5990 0.0000 0.0230 0.0005
MXNDArraySetGradState 6920 3.5270 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1384 0.5870 0.0000 0.0010 0.0004
MXAutogradBackwardEx 692 382.8160 0.5250 0.9450 0.5532
MXNDArrayGetContext 10728 4.6870 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22732 584.0750 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2768 6780.4390 0.7010 3.7500 2.4496
FullyConnected 4632 447.7640 0.0080 0.3430 0.0967
argmax 1544 30.6810 0.0050 0.0480 0.0199
_backward_pick 1384 21.9330 0.0130 0.0320 0.0158
_backward_mul_scalar 1384 6.0460 0.0030 0.0220 0.0044
DeleteVariable 58240 409.6900 0.0010 0.9010 0.0070
Flatten 1544 64.0390 0.0080 0.1120 0.0415
Activation 6176 287.1570 0.0030 1.9440 0.0465
SetValueOp 1384 8.9580 0.0050 0.0180 0.0065
pick 1384 10.9020 0.0060 0.0280 0.0079
Pooling 3088 4328.0039 0.0640 3.1720 1.4016
Convolution 3088 17613.4844 0.2260 13.2050 5.7038
CopyCPU2CPU 1544 8.7830 0.0030 0.0360 0.0057
log_softmax 1384 43.1140 0.0150 0.0550 0.0312
_backward_copy 1384 71.4880 0.0220 0.0760 0.0517
_backward_Convolution 2768 25632.8613 2.6370 12.5610 9.2604
_mul_scalar 1384 4.9540 0.0020 0.0230 0.0036
_backward_log_softmax 1384 23.6090 0.0090 0.0400 0.0171
mean 4312 82.7410 0.0080 0.0730 0.0192
_backward_mean 1384 15.1990 0.0080 0.1030 0.0110
_backward_FullyConnected 4152 796.4900 0.0280 0.5940 0.1918
WaitForVar 2928 26.1210 0.0060 0.0280 0.0089
broadcast_equal 1544 7.7160 0.0030 0.0110 0.0050
sgd_update 13840 137.1480 0.0020 0.0790 0.0099
_backward_Activation 5536 408.9830 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53682 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4638 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 692 2714.5371 0.3060 73.5850 3.9227
MXNDArrayCreateEx 773 26.6320 0.0310 0.0640 0.0345
MXNDArrayWaitAll 693 15045.2812 5.6130 25.0610 21.7104
MXNDArraySyncCopyToCPU 1466 11485.8252 0.1180 24.1980 7.8348
MXNDArrayGetDType 1466 0.5280 0.0000 0.0010 0.0004
MXNDArrayGetShape 3705 1.7300 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1546 13.5400 0.0070 0.0280 0.0088
MXNDArrayFree 17379 177.3670 0.0000 0.9140 0.0102
MXNet C API Calls 78551 565.9080 487.3580 565.9080 39.2750
MXNet C API Concurrency 157101 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1386 0.6560 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6930 3.6060 0.0000 0.0230 0.0005
MXNDArraySetGradState 6930 3.5340 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1386 0.5890 0.0000 0.0010 0.0004
MXAutogradBackwardEx 693 383.3560 0.5250 0.9450 0.5532
MXNDArrayGetContext 10742 4.6970 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22763 584.8440 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2772 6790.2402 0.7010 3.7500 2.4496
FullyConnected 4638 448.2870 0.0080 0.3430 0.0967
argmax 1546 30.7200 0.0050 0.0480 0.0199
_backward_pick 1386 21.9650 0.0130 0.0320 0.0158
_backward_mul_scalar 1386 6.0550 0.0030 0.0220 0.0044
DeleteVariable 58320 410.2600 0.0010 0.9010 0.0070
Flatten 1546 64.1220 0.0080 0.1120 0.0415
Activation 6184 287.5140 0.0030 1.9440 0.0465
SetValueOp 1386 8.9710 0.0050 0.0180 0.0065
pick 1386 10.9180 0.0060 0.0280 0.0079
Pooling 3092 4333.3589 0.0640 3.1720 1.4015
Convolution 3092 17635.6641 0.2260 13.2050 5.7036
CopyCPU2CPU 1546 8.7930 0.0030 0.0360 0.0057
log_softmax 1386 43.1740 0.0150 0.0550 0.0312
_backward_copy 1386 71.5930 0.0220 0.0760 0.0517
_backward_Convolution 2772 25669.9707 2.6370 12.5610 9.2605
_mul_scalar 1386 4.9610 0.0020 0.0230 0.0036
_backward_log_softmax 1386 23.6400 0.0090 0.0400 0.0171
mean 4318 82.8560 0.0080 0.0730 0.0192
_backward_mean 1386 15.2220 0.0080 0.1030 0.0110
_backward_FullyConnected 4158 797.5600 0.0280 0.5940 0.1918
WaitForVar 2932 26.1550 0.0060 0.0280 0.0089
broadcast_equal 1546 7.7260 0.0030 0.0110 0.0050
sgd_update 13860 137.3530 0.0020 0.0790 0.0099
_backward_Activation 5544 409.5520 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53756 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4644 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 693 2718.5649 0.3060 73.5850 3.9229
MXNDArrayCreateEx 774 26.6730 0.0310 0.0640 0.0345
MXNDArrayWaitAll 694 15067.1768 5.6130 25.0610 21.7106
MXNDArraySyncCopyToCPU 1468 11500.3066 0.1180 24.1980 7.8340
MXNDArrayGetDType 1468 0.5290 0.0000 0.0010 0.0004
MXNDArrayGetShape 3710 1.7330 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1548 13.5600 0.0070 0.0280 0.0088
MXNDArrayFree 17402 177.6250 0.0000 0.9140 0.0102
MXNet C API Calls 78658 566.0150 487.3580 566.0150 39.3285
MXNet C API Concurrency 157315 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1388 0.6580 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6940 3.6100 0.0000 0.0230 0.0005
MXNDArraySetGradState 6940 3.5400 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1388 0.5910 0.0000 0.0010 0.0004
MXAutogradBackwardEx 694 383.9550 0.5250 0.9450 0.5532
MXNDArrayGetContext 10756 4.7030 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22794 585.6700 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2776 6799.8999 0.7010 3.7500 2.4495
FullyConnected 4644 448.8500 0.0080 0.3430 0.0967
argmax 1548 30.7600 0.0050 0.0480 0.0199
_backward_pick 1388 22.0000 0.0130 0.0320 0.0159
_backward_mul_scalar 1388 6.0650 0.0030 0.0220 0.0044
DeleteVariable 58400 410.8350 0.0010 0.9010 0.0070
Flatten 1548 64.2610 0.0080 0.1120 0.0415
Activation 6192 287.9190 0.0030 1.9440 0.0465
SetValueOp 1388 8.9850 0.0050 0.0180 0.0065
pick 1388 10.9350 0.0060 0.0280 0.0079
Pooling 3096 4338.7612 0.0640 3.1720 1.4014
Convolution 3096 17658.1602 0.2260 13.2050 5.7035
CopyCPU2CPU 1548 8.8040 0.0030 0.0360 0.0057
log_softmax 1388 43.2360 0.0150 0.0550 0.0311
_backward_copy 1388 71.6980 0.0220 0.0760 0.0517
_backward_Convolution 2776 25707.0820 2.6370 12.5610 9.2605
_mul_scalar 1388 4.9680 0.0020 0.0230 0.0036
_backward_log_softmax 1388 23.6750 0.0090 0.0400 0.0171
mean 4324 82.9750 0.0080 0.0730 0.0192
_backward_mean 1388 15.2470 0.0080 0.1030 0.0110
_backward_FullyConnected 4164 798.6710 0.0280 0.5940 0.1918
WaitForVar 2936 26.1900 0.0060 0.0280 0.0089
broadcast_equal 1548 7.7370 0.0030 0.0110 0.0050
sgd_update 13880 137.5600 0.0020 0.0790 0.0099
_backward_Activation 5552 410.1540 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53830 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4650 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 694 2722.5811 0.3060 73.5850 3.9230
MXNDArrayCreateEx 775 26.7110 0.0310 0.0640 0.0345
MXNDArrayWaitAll 695 15088.9658 5.6130 25.0610 21.7107
MXNDArraySyncCopyToCPU 1470 11514.9248 0.1180 24.1980 7.8333
MXNDArrayGetDType 1470 0.5300 0.0000 0.0010 0.0004
MXNDArrayGetShape 3715 1.7350 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1550 13.5780 0.0070 0.0280 0.0088
MXNDArrayFree 17425 177.9060 0.0000 0.9140 0.0102
MXNet C API Calls 78765 566.1220 487.3580 566.1220 39.3820
MXNet C API Concurrency 157529 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1390 0.6600 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6950 3.6160 0.0000 0.0230 0.0005
MXNDArraySetGradState 6950 3.5460 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1390 0.5920 0.0000 0.0010 0.0004
MXAutogradBackwardEx 695 384.5200 0.5250 0.9450 0.5533
MXNDArrayGetContext 10770 4.7080 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22825 586.4780 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2780 6809.6362 0.7010 3.7500 2.4495
FullyConnected 4650 449.5000 0.0080 0.3430 0.0967
argmax 1550 30.8020 0.0050 0.0480 0.0199
_backward_pick 1390 22.0370 0.0130 0.0320 0.0159
_backward_mul_scalar 1390 6.0750 0.0030 0.0220 0.0044
DeleteVariable 58480 411.4620 0.0010 0.9010 0.0070
Flatten 1550 64.3880 0.0080 0.1120 0.0415
Activation 6200 288.3540 0.0030 1.9440 0.0465
SetValueOp 1390 9.0000 0.0050 0.0180 0.0065
pick 1390 10.9540 0.0060 0.0280 0.0079
Pooling 3100 4344.1162 0.0640 3.1720 1.4013
Convolution 3100 17680.4785 0.2260 13.2050 5.7034
CopyCPU2CPU 1550 8.8160 0.0030 0.0360 0.0057
log_softmax 1390 43.2980 0.0150 0.0550 0.0311
_backward_copy 1390 71.8000 0.0220 0.0760 0.0517
_backward_Convolution 2780 25743.9570 2.6370 12.5610 9.2604
_mul_scalar 1390 4.9750 0.0020 0.0230 0.0036
_backward_log_softmax 1390 23.7200 0.0090 0.0400 0.0171
mean 4330 83.1110 0.0080 0.0730 0.0192
_backward_mean 1390 15.2720 0.0080 0.1030 0.0110
_backward_FullyConnected 4170 799.8930 0.0280 0.5940 0.1918
WaitForVar 2940 26.2250 0.0060 0.0280 0.0089
broadcast_equal 1550 7.7500 0.0030 0.0110 0.0050
sgd_update 13900 137.8150 0.0020 0.0790 0.0099
_backward_Activation 5560 410.7650 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53904 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4656 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 695 2726.6021 0.3060 73.5850 3.9232
MXNDArrayCreateEx 776 26.7520 0.0310 0.0640 0.0345
MXNDArrayWaitAll 696 15110.7090 5.6130 25.0610 21.7108
MXNDArraySyncCopyToCPU 1472 11529.1475 0.1180 24.1980 7.8323
MXNDArrayGetDType 1472 0.5310 0.0000 0.0010 0.0004
MXNDArrayGetShape 3720 1.7380 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1552 13.5980 0.0070 0.0280 0.0088
MXNDArrayFree 17448 178.1570 0.0000 0.9140 0.0102
MXNet C API Calls 78872 566.2290 487.3580 566.2290 39.4355
MXNet C API Concurrency 157743 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1392 0.6600 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6960 3.6230 0.0000 0.0230 0.0005
MXNDArraySetGradState 6960 3.5490 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1392 0.5930 0.0000 0.0010 0.0004
MXAutogradBackwardEx 696 385.1360 0.5250 0.9450 0.5534
MXNDArrayGetContext 10784 4.7100 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22856 587.3300 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2784 6819.3428 0.7010 3.7500 2.4495
FullyConnected 4656 450.1420 0.0080 0.3430 0.0967
argmax 1552 30.8430 0.0050 0.0480 0.0199
_backward_pick 1392 22.0720 0.0130 0.0320 0.0159
_backward_mul_scalar 1392 6.0850 0.0030 0.0220 0.0044
DeleteVariable 58560 412.0710 0.0010 0.9010 0.0070
Flatten 1552 64.5210 0.0080 0.1120 0.0416
Activation 6208 288.7730 0.0030 1.9440 0.0465
SetValueOp 1392 9.0140 0.0050 0.0180 0.0065
pick 1392 10.9730 0.0060 0.0280 0.0079
Pooling 3104 4349.5562 0.0640 3.1720 1.4013
Convolution 3104 17702.7852 0.2260 13.2050 5.7032
CopyCPU2CPU 1552 8.8290 0.0030 0.0360 0.0057
log_softmax 1392 43.3660 0.0150 0.0550 0.0312
_backward_copy 1392 71.9040 0.0220 0.0760 0.0517
_backward_Convolution 2784 25780.7988 2.6370 12.5610 9.2603
_mul_scalar 1392 4.9830 0.0020 0.0230 0.0036
_backward_log_softmax 1392 23.7580 0.0090 0.0400 0.0171
mean 4336 83.2210 0.0080 0.0730 0.0192
_backward_mean 1392 15.2970 0.0080 0.1030 0.0110
_backward_FullyConnected 4176 801.1300 0.0280 0.5940 0.1918
WaitForVar 2944 26.2590 0.0060 0.0280 0.0089
broadcast_equal 1552 7.7610 0.0030 0.0110 0.0050
sgd_update 13920 138.0540 0.0020 0.0790 0.0099
_backward_Activation 5568 411.3690 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 53978 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4662 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 696 2730.6570 0.3060 73.5850 3.9234
MXNDArrayCreateEx 777 26.7890 0.0310 0.0640 0.0345
MXNDArrayWaitAll 697 15132.6504 5.6130 25.0610 21.7111
MXNDArraySyncCopyToCPU 1474 11543.4805 0.1180 24.1980 7.8314
MXNDArrayGetDType 1474 0.5320 0.0000 0.0010 0.0004
MXNDArrayGetShape 3725 1.7400 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1554 13.6190 0.0070 0.0280 0.0088
MXNDArrayFree 17471 178.4200 0.0000 0.9140 0.0102
MXNet C API Calls 78979 566.3360 487.3580 566.3360 39.4890
MXNet C API Concurrency 157957 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1394 0.6610 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6970 3.6290 0.0000 0.0230 0.0005
MXNDArraySetGradState 6970 3.5530 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1394 0.5940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 697 385.7350 0.5250 0.9450 0.5534
MXNDArrayGetContext 10798 4.7120 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22887 588.2180 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2788 6829.0601 0.7010 3.7500 2.4494
FullyConnected 4662 450.7270 0.0080 0.3430 0.0967
argmax 1554 30.8830 0.0050 0.0480 0.0199
_backward_pick 1394 22.1080 0.0130 0.0320 0.0159
_backward_mul_scalar 1394 6.0960 0.0030 0.0220 0.0044
DeleteVariable 58640 412.6820 0.0010 0.9010 0.0070
Flatten 1554 64.6640 0.0080 0.1120 0.0416
Activation 6216 289.1870 0.0030 1.9440 0.0465
SetValueOp 1394 9.0290 0.0050 0.0180 0.0065
pick 1394 10.9900 0.0060 0.0280 0.0079
Pooling 3108 4354.8901 0.0640 3.1720 1.4012
Convolution 3108 17725.2266 0.2260 13.2050 5.7031
CopyCPU2CPU 1554 8.8410 0.0030 0.0360 0.0057
log_softmax 1394 43.4280 0.0150 0.0550 0.0312
_backward_copy 1394 72.0070 0.0220 0.0760 0.0517
_backward_Convolution 2788 25817.9805 2.6370 12.5610 9.2604
_mul_scalar 1394 4.9900 0.0020 0.0230 0.0036
_backward_log_softmax 1394 23.7920 0.0090 0.0400 0.0171
mean 4342 83.3400 0.0080 0.0730 0.0192
_backward_mean 1394 15.3240 0.0080 0.1030 0.0110
_backward_FullyConnected 4182 802.2610 0.0280 0.5940 0.1918
WaitForVar 2948 26.2920 0.0060 0.0280 0.0089
broadcast_equal 1554 7.7730 0.0030 0.0110 0.0050
sgd_update 13940 138.2140 0.0020 0.0790 0.0099
_backward_Activation 5576 412.0550 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 54052 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4668 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 697 2734.6931 0.3060 73.5850 3.9235
MXNDArrayCreateEx 778 26.8300 0.0310 0.0640 0.0345
MXNDArrayWaitAll 698 15154.5723 5.6130 25.0610 21.7114
MXNDArraySyncCopyToCPU 1476 11557.9141 0.1180 24.1980 7.8306
MXNDArrayGetDType 1476 0.5330 0.0000 0.0010 0.0004
MXNDArrayGetShape 3730 1.7400 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1556 13.6380 0.0070 0.0280 0.0088
MXNDArrayFree 17494 178.6740 0.0000 0.9140 0.0102
MXNet C API Calls 79086 566.4430 487.3580 566.4430 39.5425
MXNet C API Concurrency 158171 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1396 0.6620 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6980 3.6340 0.0000 0.0230 0.0005
MXNDArraySetGradState 6980 3.5590 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1396 0.5940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 698 386.3120 0.5250 0.9450 0.5535
MXNDArrayGetContext 10812 4.7150 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22918 589.0430 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2792 6838.7490 0.7010 3.7500 2.4494
FullyConnected 4668 451.3000 0.0080 0.3430 0.0967
argmax 1556 30.9230 0.0050 0.0480 0.0199
_backward_pick 1396 22.1410 0.0130 0.0320 0.0159
_backward_mul_scalar 1396 6.1050 0.0030 0.0220 0.0044
DeleteVariable 58720 413.2720 0.0010 0.9010 0.0070
Flatten 1556 64.7920 0.0080 0.1120 0.0416
Activation 6224 289.5830 0.0030 1.9440 0.0465
SetValueOp 1396 9.0420 0.0050 0.0180 0.0065
pick 1396 11.0080 0.0060 0.0280 0.0079
Pooling 3112 4360.2832 0.0640 3.1720 1.4011
Convolution 3112 17747.6387 0.2260 13.2050 5.7030
CopyCPU2CPU 1556 8.8540 0.0030 0.0360 0.0057
log_softmax 1396 43.4890 0.0150 0.0550 0.0312
_backward_copy 1396 72.1090 0.0220 0.0760 0.0517
_backward_Convolution 2792 25855.1094 2.6370 12.5610 9.2604
_mul_scalar 1396 4.9980 0.0020 0.0230 0.0036
_backward_log_softmax 1396 23.8270 0.0090 0.0400 0.0171
mean 4348 83.4590 0.0080 0.0730 0.0192
_backward_mean 1396 15.3490 0.0080 0.1030 0.0110
_backward_FullyConnected 4188 803.3610 0.0280 0.5940 0.1918
WaitForVar 2952 26.3270 0.0060 0.0280 0.0089
broadcast_equal 1556 7.7850 0.0030 0.0110 0.0050
sgd_update 13960 138.3660 0.0020 0.0790 0.0099
_backward_Activation 5584 412.7130 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 54126 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4674 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 698 2738.7251 0.3060 73.5850 3.9237
MXNDArrayCreateEx 779 26.8740 0.0310 0.0640 0.0345
MXNDArrayWaitAll 699 15176.3652 5.6130 25.0610 21.7115
MXNDArraySyncCopyToCPU 1478 11572.4346 0.1180 24.1980 7.8298
MXNDArrayGetDType 1478 0.5340 0.0000 0.0010 0.0004
MXNDArrayGetShape 3735 1.7400 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1558 13.6580 0.0070 0.0280 0.0088
MXNDArrayFree 17517 178.9400 0.0000 0.9140 0.0102
MXNet C API Calls 79193 566.5500 487.3580 566.5500 39.5960
MXNet C API Concurrency 158385 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1398 0.6620 0.0000 0.0040 0.0005
MXNDArrayGetGradState 6990 3.6420 0.0000 0.0230 0.0005
MXNDArraySetGradState 6990 3.5620 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1398 0.5940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 699 386.8910 0.5250 0.9450 0.5535
MXNDArrayGetContext 10826 4.7220 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22949 589.8860 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2796 6848.4302 0.7010 3.7500 2.4494
FullyConnected 4674 451.9040 0.0080 0.3430 0.0967
argmax 1558 30.9630 0.0050 0.0480 0.0199
_backward_pick 1398 22.1770 0.0130 0.0320 0.0159
_backward_mul_scalar 1398 6.1140 0.0030 0.0220 0.0044
DeleteVariable 58800 413.8940 0.0010 0.9010 0.0070
Flatten 1558 64.9330 0.0080 0.1120 0.0417
Activation 6232 289.9970 0.0030 1.9440 0.0465
SetValueOp 1398 9.0540 0.0050 0.0180 0.0065
pick 1398 11.0250 0.0060 0.0280 0.0079
Pooling 3116 4365.6992 0.0640 3.1720 1.4011
Convolution 3116 17770.1172 0.2260 13.2050 5.7029
CopyCPU2CPU 1558 8.8670 0.0030 0.0360 0.0057
log_softmax 1398 43.5510 0.0150 0.0550 0.0312
_backward_copy 1398 72.2120 0.0220 0.0760 0.0517
_backward_Convolution 2796 25892.0469 2.6370 12.5610 9.2604
_mul_scalar 1398 5.0050 0.0020 0.0230 0.0036
_backward_log_softmax 1398 23.8610 0.0090 0.0400 0.0171
mean 4354 83.5790 0.0080 0.0730 0.0192
_backward_mean 1398 15.3730 0.0080 0.1030 0.0110
_backward_FullyConnected 4194 804.4630 0.0280 0.5940 0.1918
WaitForVar 2956 26.3630 0.0060 0.0280 0.0089
broadcast_equal 1558 7.7960 0.0030 0.0110 0.0050
sgd_update 13980 138.5770 0.0020 0.0790 0.0099
_backward_Activation 5592 413.3210 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 54200 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4680 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 699 2742.8259 0.3060 73.5850 3.9239
MXNDArrayCreateEx 780 26.9220 0.0310 0.0640 0.0345
MXNDArrayWaitAll 700 15198.1992 5.6130 25.0610 21.7117
MXNDArraySyncCopyToCPU 1480 11586.9883 0.1180 24.1980 7.8290
MXNDArrayGetDType 1480 0.5350 0.0000 0.0010 0.0004
MXNDArrayGetShape 3740 1.7420 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1560 13.6810 0.0070 0.0280 0.0088
MXNDArrayFree 17540 179.2260 0.0000 0.9140 0.0102
MXNet C API Calls 79300 566.6570 487.3580 566.6570 39.6495
MXNet C API Concurrency 158599 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1400 0.6640 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7000 3.6460 0.0000 0.0230 0.0005
MXNDArraySetGradState 7000 3.5680 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1400 0.5940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 700 387.4750 0.5250 0.9450 0.5535
MXNDArrayGetContext 10840 4.7280 0.0000 0.0070 0.0004
MXImperativeInvokeEx 22980 590.7550 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2800 6858.1831 0.7010 3.7500 2.4494
FullyConnected 4680 452.4970 0.0080 0.3430 0.0967
argmax 1560 31.0030 0.0050 0.0480 0.0199
_backward_pick 1400 22.2130 0.0130 0.0320 0.0159
_backward_mul_scalar 1400 6.1240 0.0030 0.0220 0.0044
DeleteVariable 58880 414.5030 0.0010 0.9010 0.0070
Flatten 1560 65.0960 0.0080 0.1120 0.0417
Activation 6240 290.4540 0.0030 1.9440 0.0465
SetValueOp 1400 9.0690 0.0050 0.0180 0.0065
pick 1400 11.0440 0.0060 0.0280 0.0079
Pooling 3120 4371.0571 0.0640 3.1720 1.4010
Convolution 3120 17792.7578 0.2260 13.2050 5.7028
CopyCPU2CPU 1560 8.8800 0.0030 0.0360 0.0057
log_softmax 1400 43.6140 0.0150 0.0550 0.0312
_backward_copy 1400 72.3150 0.0220 0.0760 0.0517
_backward_Convolution 2800 25928.9395 2.6370 12.5610 9.2603
_mul_scalar 1400 5.0140 0.0020 0.0230 0.0036
_backward_log_softmax 1400 23.8960 0.0090 0.0400 0.0171
mean 4360 83.7010 0.0080 0.0730 0.0192
_backward_mean 1400 15.3990 0.0080 0.1030 0.0110
_backward_FullyConnected 4200 805.5660 0.0280 0.5940 0.1918
WaitForVar 2960 26.3980 0.0060 0.0280 0.0089
broadcast_equal 1560 7.8080 0.0030 0.0110 0.0050
sgd_update 14000 138.8090 0.0020 0.0790 0.0099
_backward_Activation 5600 413.9650 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 54274 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4686 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 700 2743.4021 0.3060 73.5850 3.9191
MXNDArrayCreateEx 781 26.9630 0.0310 0.0640 0.0345
MXNDArrayWaitAll 701 15220.1426 5.6130 25.0610 21.7120
MXNDArraySyncCopyToCPU 1482 11601.3291 0.1180 24.1980 7.8282
MXNDArrayGetDType 1482 0.5350 0.0000 0.0010 0.0004
MXNDArrayGetShape 3745 1.7440 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1562 13.7020 0.0070 0.0280 0.0088
MXNDArrayFree 17563 179.4940 0.0000 0.9140 0.0102
MXNet C API Calls 79407 566.7640 487.3580 566.7640 39.7030
MXNet C API Concurrency 158813 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1402 0.6660 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7010 3.6510 0.0000 0.0230 0.0005
MXNDArraySetGradState 7010 3.5750 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1402 0.5950 0.0000 0.0010 0.0004
MXAutogradBackwardEx 701 388.0600 0.5250 0.9450 0.5536
MXNDArrayGetContext 10854 4.7330 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23011 591.5940 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2804 6867.9722 0.7010 3.7500 2.4493
FullyConnected 4686 453.0900 0.0080 0.3430 0.0967
argmax 1562 31.0420 0.0050 0.0480 0.0199
_backward_pick 1402 22.2460 0.0130 0.0320 0.0159
_backward_mul_scalar 1402 6.1350 0.0030 0.0220 0.0044
DeleteVariable 58960 415.1210 0.0010 0.9010 0.0070
Flatten 1562 65.2370 0.0080 0.1120 0.0418
Activation 6248 290.8360 0.0030 1.9440 0.0465
SetValueOp 1402 9.0820 0.0050 0.0180 0.0065
pick 1402 11.0620 0.0060 0.0280 0.0079
Pooling 3124 4376.4561 0.0640 3.1720 1.4009
Convolution 3124 17815.0137 0.2260 13.2050 5.7026
CopyCPU2CPU 1562 8.8930 0.0030 0.0360 0.0057
log_softmax 1402 43.6770 0.0150 0.0550 0.0312
_backward_copy 1402 72.4160 0.0220 0.0760 0.0517
_backward_Convolution 2804 25966.0352 2.6370 12.5610 9.2604
_mul_scalar 1402 5.0210 0.0020 0.0230 0.0036
_backward_log_softmax 1402 23.9290 0.0090 0.0400 0.0171
mean 4366 83.8160 0.0080 0.0730 0.0192
_backward_mean 1402 15.4240 0.0080 0.1030 0.0110
_backward_FullyConnected 4206 806.6750 0.0280 0.5940 0.1918
WaitForVar 2964 26.4320 0.0060 0.0280 0.0089
broadcast_equal 1562 7.8190 0.0030 0.0110 0.0050
sgd_update 14020 138.9590 0.0020 0.0790 0.0099
_backward_Activation 5608 414.6400 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 54348 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4692 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 701 2747.4519 0.3060 73.5850 3.9193
MXNDArrayCreateEx 782 27.0110 0.0310 0.0640 0.0345
MXNDArrayWaitAll 702 15242.0908 5.6130 25.0610 21.7124
MXNDArraySyncCopyToCPU 1484 11615.8457 0.1180 24.1980 7.8274
MXNDArrayGetDType 1484 0.5360 0.0000 0.0010 0.0004
MXNDArrayGetShape 3750 1.7480 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1564 13.7230 0.0070 0.0280 0.0088
MXNDArrayFree 17586 179.7630 0.0000 0.9140 0.0102
MXNet C API Calls 79514 566.8710 487.3580 566.8710 39.7565
MXNet C API Concurrency 159027 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1404 0.6680 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7020 3.6570 0.0000 0.0230 0.0005
MXNDArraySetGradState 7020 3.5810 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1404 0.5950 0.0000 0.0010 0.0004
MXAutogradBackwardEx 702 388.6500 0.5250 0.9450 0.5536
MXNDArrayGetContext 10868 4.7380 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23042 592.4910 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2808 6877.6748 0.7010 3.7500 2.4493
FullyConnected 4692 453.6880 0.0080 0.3430 0.0967
argmax 1564 31.0820 0.0050 0.0480 0.0199
_backward_pick 1404 22.2820 0.0130 0.0320 0.0159
_backward_mul_scalar 1404 6.1440 0.0030 0.0220 0.0044
DeleteVariable 59040 415.7240 0.0010 0.9010 0.0070
Flatten 1564 65.3970 0.0080 0.1120 0.0418
Activation 6256 291.2870 0.0030 1.9440 0.0466
SetValueOp 1404 9.0980 0.0050 0.0180 0.0065
pick 1404 11.0820 0.0060 0.0280 0.0079
Pooling 3128 4381.8262 0.0640 3.1720 1.4008
Convolution 3128 17837.6797 0.2260 13.2050 5.7026
CopyCPU2CPU 1564 8.9060 0.0030 0.0360 0.0057
log_softmax 1404 43.7420 0.0150 0.0550 0.0312
_backward_copy 1404 72.5190 0.0220 0.0760 0.0517
_backward_Convolution 2808 26003.1875 2.6370 12.5610 9.2604
_mul_scalar 1404 5.0280 0.0020 0.0230 0.0036
_backward_log_softmax 1404 23.9640 0.0090 0.0400 0.0171
mean 4372 83.9370 0.0080 0.0730 0.0192
_backward_mean 1404 15.4500 0.0080 0.1030 0.0110
_backward_FullyConnected 4212 807.7850 0.0280 0.5940 0.1918
WaitForVar 2968 26.4670 0.0060 0.0280 0.0089
broadcast_equal 1564 7.8320 0.0030 0.0110 0.0050
sgd_update 14040 139.1100 0.0020 0.0790 0.0099
_backward_Activation 5616 415.3490 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 54422 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4698 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 702 2751.4700 0.3060 73.5850 3.9195
MXNDArrayCreateEx 783 27.0510 0.0310 0.0640 0.0345
MXNDArrayWaitAll 703 15263.8057 5.6130 25.0610 21.7124
MXNDArraySyncCopyToCPU 1486 11630.1406 0.1180 24.1980 7.8265
MXNDArrayGetDType 1486 0.5370 0.0000 0.0010 0.0004
MXNDArrayGetShape 3755 1.7510 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1566 13.7420 0.0070 0.0280 0.0088
MXNDArrayFree 17609 180.0270 0.0000 0.9140 0.0102
MXNet C API Calls 79621 566.9780 487.3580 566.9780 39.8100
MXNet C API Concurrency 159241 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1406 0.6680 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7030 3.6590 0.0000 0.0230 0.0005
MXNDArraySetGradState 7030 3.5860 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1406 0.5950 0.0000 0.0010 0.0004
MXAutogradBackwardEx 703 389.2280 0.5250 0.9450 0.5537
MXNDArrayGetContext 10882 4.7460 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23073 593.3040 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2812 6887.2759 0.7010 3.7500 2.4492
FullyConnected 4698 454.2270 0.0080 0.3430 0.0967
argmax 1566 31.1220 0.0050 0.0480 0.0199
_backward_pick 1406 22.3140 0.0130 0.0320 0.0159
_backward_mul_scalar 1406 6.1530 0.0030 0.0220 0.0044
DeleteVariable 59120 416.3030 0.0010 0.9010 0.0070
Flatten 1566 65.4950 0.0080 0.1120 0.0418
Activation 6264 291.6480 0.0030 1.9440 0.0466
SetValueOp 1406 9.1110 0.0050 0.0180 0.0065
pick 1406 11.0980 0.0060 0.0280 0.0079
Pooling 3132 4387.1738 0.0640 3.1720 1.4008
Convolution 3132 17859.9746 0.2260 13.2050 5.7024
CopyCPU2CPU 1566 8.9180 0.0030 0.0360 0.0057
log_softmax 1406 43.8030 0.0150 0.0550 0.0312
_backward_copy 1406 72.6230 0.0220 0.0760 0.0517
_backward_Convolution 2812 26040.0137 2.6370 12.5610 9.2603
_mul_scalar 1406 5.0350 0.0020 0.0230 0.0036
_backward_log_softmax 1406 23.9980 0.0090 0.0400 0.0171
mean 4378 84.0520 0.0080 0.0730 0.0192
_backward_mean 1406 15.4740 0.0080 0.1030 0.0110
_backward_FullyConnected 4218 808.8740 0.0280 0.5940 0.1918
WaitForVar 2972 26.5000 0.0060 0.0280 0.0089
broadcast_equal 1566 7.8440 0.0030 0.0110 0.0050
sgd_update 14060 139.3140 0.0020 0.0790 0.0099
_backward_Activation 5624 415.9000 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 54496 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4704 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 703 2755.4050 0.3060 73.5850 3.9195
MXNDArrayCreateEx 784 27.0860 0.0310 0.0640 0.0345
MXNDArrayWaitAll 704 15285.4707 5.6130 25.0610 21.7123
MXNDArraySyncCopyToCPU 1488 11644.4414 0.1180 24.1980 7.8256
MXNDArrayGetDType 1488 0.5380 0.0000 0.0010 0.0004
MXNDArrayGetShape 3760 1.7540 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1568 13.7590 0.0070 0.0280 0.0088
MXNDArrayFree 17632 180.2740 0.0000 0.9140 0.0102
MXNet C API Calls 79728 567.0850 487.3580 567.0850 39.8635
MXNet C API Concurrency 159455 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1408 0.6680 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7040 3.6660 0.0000 0.0230 0.0005
MXNDArraySetGradState 7040 3.5910 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1408 0.5960 0.0000 0.0010 0.0004
MXAutogradBackwardEx 704 389.7810 0.5250 0.9450 0.5537
MXNDArrayGetContext 10896 4.7470 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23104 594.0810 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2816 6896.9199 0.7010 3.7500 2.4492
FullyConnected 4704 454.7600 0.0080 0.3430 0.0967
argmax 1568 31.1610 0.0050 0.0480 0.0199
_backward_pick 1408 22.3450 0.0130 0.0320 0.0159
_backward_mul_scalar 1408 6.1620 0.0030 0.0220 0.0044
DeleteVariable 59200 416.8590 0.0010 0.9010 0.0070
Flatten 1568 65.5750 0.0080 0.1120 0.0418
Activation 6272 291.9980 0.0030 1.9440 0.0466
SetValueOp 1408 9.1230 0.0050 0.0180 0.0065
pick 1408 11.1140 0.0060 0.0280 0.0079
Pooling 3136 4392.5200 0.0640 3.1720 1.4007
Convolution 3136 17882.0742 0.2260 13.2050 5.7022
CopyCPU2CPU 1568 8.9290 0.0030 0.0360 0.0057
log_softmax 1408 43.8630 0.0150 0.0550 0.0312
_backward_copy 1408 72.7260 0.0220 0.0760 0.0517
_backward_Convolution 2816 26076.7637 2.6370 12.5610 9.2602
_mul_scalar 1408 5.0420 0.0020 0.0230 0.0036
_backward_log_softmax 1408 24.0300 0.0090 0.0400 0.0171
mean 4384 84.1690 0.0080 0.0730 0.0192
_backward_mean 1408 15.4960 0.0080 0.1030 0.0110
_backward_FullyConnected 4224 809.9400 0.0280 0.5940 0.1917
WaitForVar 2976 26.5360 0.0060 0.0280 0.0089
broadcast_equal 1568 7.8540 0.0030 0.0110 0.0050
sgd_update 14080 139.5090 0.0020 0.0790 0.0099
_backward_Activation 5632 416.4470 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 54570 55478.5859 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4710 1055.0400 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 704 2759.3000 0.3060 73.5850 3.9195
MXNDArrayCreateEx 785 27.1190 0.0310 0.0640 0.0345
MXNDArrayWaitAll 705 15291.1992 5.6130 25.0610 21.6896
MXNDArraySyncCopyToCPU 1490 11651.1533 0.1180 24.1980 7.8196
MXNDArrayGetDType 1490 0.5390 0.0000 0.0010 0.0004
MXNDArrayGetShape 3765 1.7550 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1570 13.7750 0.0070 0.0280 0.0088
MXNDArrayFree 17655 180.4960 0.0000 0.9140 0.0102
MXNet C API Calls 79835 567.1920 487.3580 567.1920 39.9170
MXNet C API Concurrency 159669 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1410 0.6690 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7050 3.6700 0.0000 0.0230 0.0005
MXNDArraySetGradState 7050 3.5960 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1410 0.5960 0.0000 0.0010 0.0004
MXAutogradBackwardEx 705 390.3430 0.5250 0.9450 0.5537
MXNDArrayGetContext 10910 4.7530 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23135 594.8500 0.0090 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2820 6900.7100 0.7010 3.7500 2.4471
FullyConnected 4710 455.0720 0.0080 0.3430 0.0966
argmax 1570 31.1820 0.0050 0.0480 0.0199
_backward_pick 1410 22.3710 0.0130 0.0320 0.0159
_backward_mul_scalar 1410 6.1690 0.0030 0.0220 0.0044
DeleteVariable 59280 417.3550 0.0010 0.9010 0.0070
Flatten 1570 65.6140 0.0080 0.1120 0.0418
Activation 6280 292.1680 0.0030 1.9440 0.0465
SetValueOp 1410 9.1340 0.0050 0.0180 0.0065
pick 1410 11.1260 0.0060 0.0280 0.0079
Pooling 3140 4394.5439 0.0640 3.1720 1.3995
Convolution 3140 17890.4980 0.2260 13.2050 5.6976
CopyCPU2CPU 1570 8.9390 0.0030 0.0360 0.0057
log_softmax 1410 43.8940 0.0150 0.0550 0.0311
_backward_copy 1410 72.7720 0.0220 0.0760 0.0516
_backward_Convolution 2820 26090.8281 2.6240 12.5610 9.2521
_mul_scalar 1410 5.0480 0.0020 0.0230 0.0036
_backward_log_softmax 1410 24.0490 0.0090 0.0400 0.0171
mean 4390 84.2610 0.0080 0.0730 0.0192
_backward_mean 1410 15.5140 0.0080 0.1030 0.0110
_backward_FullyConnected 4230 810.4410 0.0280 0.5940 0.1916
WaitForVar 2980 26.5680 0.0060 0.0280 0.0089
broadcast_equal 1570 7.8620 0.0030 0.0110 0.0050
sgd_update 14100 139.6570 0.0020 0.0790 0.0099
_backward_Activation 5640 416.6720 0.0070 0.3900 0.0739
Epoch 0: loss 0.173, train_acc 0.935, test acc 0.905, in 11.7 sec
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 55844 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4956 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 705 2763.1851 0.3060 73.5850 3.9194
MXNDArrayCreateEx 826 28.4540 0.0310 0.0640 0.0344
MXNDArrayWaitAll 706 15313.1426 5.6130 25.0610 21.6900
MXNDArraySyncCopyToCPU 1532 12225.4316 0.0600 24.1980 7.9800
MXNDArrayGetDType 1532 0.5600 0.0000 0.0010 0.0004
MXNDArrayGetShape 3890 1.8240 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1652 14.4670 0.0070 0.0280 0.0088
MXNDArrayFree 18398 188.9330 0.0000 0.9140 0.0103
MXNet C API Calls 82142 569.4990 487.3580 569.4990 41.0705
MXNet C API Concurrency 164283 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1412 0.6690 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7060 3.6760 0.0000 0.0230 0.0005
MXNDArraySetGradState 7060 3.6010 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1412 0.5980 0.0000 0.0010 0.0004
MXAutogradBackwardEx 706 390.9620 0.5250 0.9450 0.5538
MXNDArrayGetContext 11444 4.9840 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23806 609.1990 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2824 6910.5000 0.7010 3.7500 2.4471
FullyConnected 4956 477.6760 0.0080 0.3430 0.0964
argmax 1652 32.7140 0.0050 0.0480 0.0198
_backward_pick 1412 22.4080 0.0130 0.0320 0.0159
_backward_mul_scalar 1412 6.1800 0.0030 0.0220 0.0044
DeleteVariable 60800 435.9370 0.0010 0.9010 0.0072
Flatten 1652 69.4410 0.0080 0.1120 0.0420
Activation 6608 306.7800 0.0030 1.9440 0.0464
SetValueOp 1412 9.1480 0.0050 0.0180 0.0065
pick 1412 11.1450 0.0060 0.0280 0.0079
Pooling 3304 4623.5332 0.0640 3.1720 1.3994
Convolution 3304 18839.4160 0.2260 13.2050 5.7020
CopyCPU2CPU 1652 9.6360 0.0030 0.0360 0.0058
log_softmax 1412 43.9710 0.0150 0.0550 0.0311
_backward_copy 1412 72.8790 0.0220 0.0760 0.0516
_backward_Convolution 2824 26127.9277 2.6240 12.5610 9.2521
_mul_scalar 1412 5.0560 0.0020 0.0230 0.0036
_backward_log_softmax 1412 24.0860 0.0090 0.0400 0.0171
mean 4476 85.6040 0.0080 0.0730 0.0191
_backward_mean 1412 15.5380 0.0080 0.1030 0.0110
_backward_FullyConnected 4236 811.6700 0.0280 0.5940 0.1916
WaitForVar 3064 27.2300 0.0060 0.0280 0.0089
broadcast_equal 1652 8.2980 0.0030 0.0110 0.0050
sgd_update 14120 139.8790 0.0020 0.0790 0.0099
_backward_Activation 5648 417.3340 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 55918 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4962 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 706 2836.7720 0.3060 73.5870 4.0181
MXNDArrayCreateEx 827 28.4880 0.0310 0.0640 0.0344
MXNDArrayWaitAll 707 15334.8721 5.6130 25.0610 21.6901
MXNDArraySyncCopyToCPU 1534 12239.9385 0.0600 24.1980 7.9791
MXNDArrayGetDType 1534 0.5600 0.0000 0.0010 0.0004
MXNDArrayGetShape 3895 1.8280 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1654 14.4850 0.0070 0.0280 0.0088
MXNDArrayFree 18421 189.1690 0.0000 0.9140 0.0103
MXNet C API Calls 82249 569.6060 487.3580 569.6060 41.1240
MXNet C API Concurrency 164497 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1414 0.6700 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7070 3.6810 0.0000 0.0230 0.0005
MXNDArraySetGradState 7070 3.6090 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1414 0.5990 0.0000 0.0010 0.0004
MXAutogradBackwardEx 707 391.5400 0.5250 0.9450 0.5538
MXNDArrayGetContext 11458 4.9930 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23837 610.0290 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2828 6920.2930 0.7010 3.7500 2.4471
FullyConnected 4962 478.2540 0.0080 0.3430 0.0964
argmax 1654 32.7540 0.0050 0.0480 0.0198
_backward_pick 1414 22.4440 0.0130 0.0320 0.0159
_backward_mul_scalar 1414 6.1910 0.0030 0.0220 0.0044
DeleteVariable 60880 436.4910 0.0010 0.9010 0.0072
Flatten 1654 69.5780 0.0080 0.1120 0.0421
Activation 6616 307.2060 0.0030 1.9440 0.0464
SetValueOp 1414 9.1620 0.0050 0.0180 0.0065
pick 1414 11.1620 0.0060 0.0280 0.0079
Pooling 3308 4628.9248 0.0640 3.1720 1.3993
Convolution 3308 18861.8125 0.2260 13.2050 5.7019
CopyCPU2CPU 1654 9.6470 0.0030 0.0360 0.0058
log_softmax 1414 44.0340 0.0150 0.0550 0.0311
_backward_copy 1414 72.9850 0.0220 0.0760 0.0516
_backward_Convolution 2828 26164.6582 2.6240 12.5610 9.2520
_mul_scalar 1414 5.0640 0.0020 0.0230 0.0036
_backward_log_softmax 1414 24.1190 0.0090 0.0400 0.0171
mean 4482 85.7260 0.0080 0.0730 0.0191
_backward_mean 1414 15.5630 0.0080 0.1030 0.0110
_backward_FullyConnected 4242 812.7680 0.0280 0.5940 0.1916
WaitForVar 3068 27.2640 0.0060 0.0280 0.0089
broadcast_equal 1654 8.3090 0.0030 0.0110 0.0050
sgd_update 14140 140.0870 0.0020 0.0790 0.0099
_backward_Activation 5656 417.9330 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 55992 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4968 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 707 2840.6970 0.3060 73.5870 4.0180
MXNDArrayCreateEx 828 28.5210 0.0310 0.0640 0.0344
MXNDArrayWaitAll 708 15356.5859 5.6130 25.0610 21.6901
MXNDArraySyncCopyToCPU 1536 12254.4648 0.0600 24.1980 7.9782
MXNDArrayGetDType 1536 0.5600 0.0000 0.0010 0.0004
MXNDArrayGetShape 3900 1.8290 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1656 14.5020 0.0070 0.0280 0.0088
MXNDArrayFree 18444 189.4010 0.0000 0.9140 0.0103
MXNet C API Calls 82356 569.7130 487.3580 569.7130 41.1775
MXNet C API Concurrency 164711 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1416 0.6710 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7080 3.6880 0.0000 0.0230 0.0005
MXNDArraySetGradState 7080 3.6150 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1416 0.6000 0.0000 0.0010 0.0004
MXAutogradBackwardEx 708 392.0830 0.5250 0.9450 0.5538
MXNDArrayGetContext 11472 5.0000 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23868 610.8010 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2832 6930.1528 0.7010 3.7500 2.4471
FullyConnected 4968 478.8660 0.0080 0.3430 0.0964
argmax 1656 32.7950 0.0050 0.0480 0.0198
_backward_pick 1416 22.4780 0.0130 0.0320 0.0159
_backward_mul_scalar 1416 6.1990 0.0030 0.0220 0.0044
DeleteVariable 60960 437.0850 0.0010 0.9010 0.0072
Flatten 1656 69.6590 0.0080 0.1120 0.0421
Activation 6624 307.5780 0.0030 1.9440 0.0464
SetValueOp 1416 9.1740 0.0050 0.0180 0.0065
pick 1416 11.1770 0.0060 0.0280 0.0079
Pooling 3312 4634.2930 0.0640 3.1720 1.3992
Convolution 3312 18883.9473 0.2260 13.2050 5.7017
CopyCPU2CPU 1656 9.6570 0.0030 0.0360 0.0058
log_softmax 1416 44.0940 0.0150 0.0550 0.0311
_backward_copy 1416 73.0900 0.0220 0.0760 0.0516
_backward_Convolution 2832 26201.3184 2.6240 12.5610 9.2519
_mul_scalar 1416 5.0700 0.0020 0.0230 0.0036
_backward_log_softmax 1416 24.1610 0.0090 0.0400 0.0171
mean 4488 85.8330 0.0080 0.0730 0.0191
_backward_mean 1416 15.5870 0.0080 0.1030 0.0110
_backward_FullyConnected 4248 813.9410 0.0280 0.5940 0.1916
WaitForVar 3072 27.2990 0.0060 0.0280 0.0089
broadcast_equal 1656 8.3200 0.0030 0.0110 0.0050
sgd_update 14160 140.3200 0.0020 0.0790 0.0099
_backward_Activation 5664 418.4870 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56066 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4974 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 708 2844.6121 0.3060 73.5870 4.0178
MXNDArrayCreateEx 829 28.5550 0.0310 0.0640 0.0344
MXNDArrayWaitAll 709 15378.4180 5.6130 25.0610 21.6903
MXNDArraySyncCopyToCPU 1538 12268.7402 0.0600 24.1980 7.9771
MXNDArrayGetDType 1538 0.5610 0.0000 0.0010 0.0004
MXNDArrayGetShape 3905 1.8300 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1658 14.5200 0.0070 0.0280 0.0088
MXNDArrayFree 18467 189.6320 0.0000 0.9140 0.0103
MXNet C API Calls 82463 569.8200 487.3580 569.8200 41.2310
MXNet C API Concurrency 164925 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1418 0.6710 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7090 3.6950 0.0000 0.0230 0.0005
MXNDArraySetGradState 7090 3.6190 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1418 0.6020 0.0000 0.0010 0.0004
MXAutogradBackwardEx 709 392.6600 0.5250 0.9450 0.5538
MXNDArrayGetContext 11486 5.0050 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23899 611.5920 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2836 6939.8271 0.7010 3.7500 2.4470
FullyConnected 4974 479.4270 0.0080 0.3430 0.0964
argmax 1658 32.8340 0.0050 0.0480 0.0198
_backward_pick 1418 22.5110 0.0130 0.0320 0.0159
_backward_mul_scalar 1418 6.2070 0.0030 0.0220 0.0044
DeleteVariable 61040 437.6070 0.0010 0.9010 0.0072
Flatten 1658 69.7380 0.0080 0.1120 0.0421
Activation 6632 307.9460 0.0030 1.9440 0.0464
SetValueOp 1418 9.1870 0.0050 0.0180 0.0065
pick 1418 11.1930 0.0060 0.0280 0.0079
Pooling 3316 4639.6821 0.0640 3.1720 1.3992
Convolution 3316 18906.1445 0.2260 13.2050 5.7015
CopyCPU2CPU 1658 9.6660 0.0030 0.0360 0.0058
log_softmax 1418 44.1540 0.0150 0.0550 0.0311
_backward_copy 1418 73.1960 0.0220 0.0760 0.0516
_backward_Convolution 2836 26238.3594 2.6240 12.5610 9.2519
_mul_scalar 1418 5.0770 0.0020 0.0230 0.0036
_backward_log_softmax 1418 24.1940 0.0090 0.0400 0.0171
mean 4494 85.9480 0.0080 0.0730 0.0191
_backward_mean 1418 15.6090 0.0080 0.1030 0.0110
_backward_FullyConnected 4254 815.0080 0.0280 0.5940 0.1916
WaitForVar 3076 27.3330 0.0060 0.0280 0.0089
broadcast_equal 1658 8.3310 0.0030 0.0110 0.0050
sgd_update 14180 140.5260 0.0020 0.0790 0.0099
_backward_Activation 5672 419.0580 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56140 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4980 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 709 2848.5039 0.3060 73.5870 4.0176
MXNDArrayCreateEx 830 28.5890 0.0310 0.0640 0.0344
MXNDArrayWaitAll 710 15400.3047 5.6130 25.0610 21.6906
MXNDArraySyncCopyToCPU 1540 12283.2441 0.0600 24.1980 7.9761
MXNDArrayGetDType 1540 0.5610 0.0000 0.0010 0.0004
MXNDArrayGetShape 3910 1.8320 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1660 14.5370 0.0070 0.0280 0.0088
MXNDArrayFree 18490 189.8630 0.0000 0.9140 0.0103
MXNet C API Calls 82570 569.9270 487.3580 569.9270 41.2845
MXNet C API Concurrency 165139 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1420 0.6720 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7100 3.7000 0.0000 0.0230 0.0005
MXNDArraySetGradState 7100 3.6230 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1420 0.6020 0.0000 0.0010 0.0004
MXAutogradBackwardEx 710 393.1920 0.5250 0.9450 0.5538
MXNDArrayGetContext 11500 5.0140 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23930 612.3650 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2840 6949.6030 0.7010 3.7500 2.4470
FullyConnected 4980 479.9780 0.0080 0.3430 0.0964
argmax 1660 32.8710 0.0050 0.0480 0.0198
_backward_pick 1420 22.5420 0.0130 0.0320 0.0159
_backward_mul_scalar 1420 6.2160 0.0030 0.0220 0.0044
DeleteVariable 61120 438.1490 0.0010 0.9010 0.0072
Flatten 1660 69.8170 0.0080 0.1120 0.0421
Activation 6640 308.3170 0.0030 1.9440 0.0464
SetValueOp 1420 9.2010 0.0050 0.0180 0.0065
pick 1420 11.2080 0.0060 0.0280 0.0079
Pooling 3320 4645.0518 0.0640 3.1720 1.3991
Convolution 3320 18928.4258 0.2260 13.2050 5.7013
CopyCPU2CPU 1660 9.6760 0.0030 0.0360 0.0058
log_softmax 1420 44.2120 0.0150 0.0550 0.0311
_backward_copy 1420 73.3030 0.0220 0.0760 0.0516
_backward_Convolution 2840 26275.4512 2.6240 12.5610 9.2519
_mul_scalar 1420 5.0840 0.0020 0.0230 0.0036
_backward_log_softmax 1420 24.2250 0.0090 0.0400 0.0171
mean 4500 86.0880 0.0080 0.0730 0.0191
_backward_mean 1420 15.6300 0.0080 0.1030 0.0110
_backward_FullyConnected 4260 816.0790 0.0280 0.5940 0.1916
WaitForVar 3080 27.3670 0.0060 0.0280 0.0089
broadcast_equal 1660 8.3410 0.0030 0.0110 0.0050
sgd_update 14200 140.7340 0.0020 0.0790 0.0099
_backward_Activation 5680 419.6310 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56214 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4986 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 710 2852.4050 0.3060 73.5870 4.0175
MXNDArrayCreateEx 831 28.6220 0.0310 0.0640 0.0344
MXNDArrayWaitAll 711 15422.0469 5.6130 25.0610 21.6906
MXNDArraySyncCopyToCPU 1542 12297.5312 0.0600 24.1980 7.9751
MXNDArrayGetDType 1542 0.5610 0.0000 0.0010 0.0004
MXNDArrayGetShape 3915 1.8340 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1662 14.5540 0.0070 0.0280 0.0088
MXNDArrayFree 18513 190.0890 0.0000 0.9140 0.0103
MXNet C API Calls 82677 570.0340 487.3580 570.0340 41.3380
MXNet C API Concurrency 165353 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1422 0.6720 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7110 3.7080 0.0000 0.0230 0.0005
MXNDArraySetGradState 7110 3.6290 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1422 0.6020 0.0000 0.0010 0.0004
MXAutogradBackwardEx 711 393.7450 0.5250 0.9450 0.5538
MXNDArrayGetContext 11514 5.0170 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23961 613.1610 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2844 6959.3359 0.7010 3.7500 2.4470
FullyConnected 4986 480.5100 0.0080 0.3430 0.0964
argmax 1662 32.9090 0.0050 0.0480 0.0198
_backward_pick 1422 22.5730 0.0130 0.0320 0.0159
_backward_mul_scalar 1422 6.2250 0.0030 0.0220 0.0044
DeleteVariable 61200 438.6670 0.0010 0.9010 0.0072
Flatten 1662 69.8940 0.0080 0.1120 0.0421
Activation 6648 308.6860 0.0030 1.9440 0.0464
SetValueOp 1422 9.2140 0.0050 0.0180 0.0065
pick 1422 11.2240 0.0060 0.0280 0.0079
Pooling 3324 4650.3960 0.0640 3.1720 1.3990
Convolution 3324 18950.6484 0.2260 13.2050 5.7012
CopyCPU2CPU 1662 9.6860 0.0030 0.0360 0.0058
log_softmax 1422 44.2730 0.0150 0.0550 0.0311
_backward_copy 1422 73.4080 0.0220 0.0760 0.0516
_backward_Convolution 2844 26312.2480 2.6240 12.5610 9.2518
_mul_scalar 1422 5.0900 0.0020 0.0230 0.0036
_backward_log_softmax 1422 24.2570 0.0090 0.0400 0.0171
mean 4506 86.2120 0.0080 0.0730 0.0191
_backward_mean 1422 15.6520 0.0080 0.1030 0.0110
_backward_FullyConnected 4266 817.1560 0.0280 0.5940 0.1916
WaitForVar 3084 27.4010 0.0060 0.0280 0.0089
broadcast_equal 1662 8.3510 0.0030 0.0110 0.0050
sgd_update 14220 140.9430 0.0020 0.0790 0.0099
_backward_Activation 5688 420.1880 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56288 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4992 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 711 2856.3010 0.3060 73.5870 4.0173
MXNDArrayCreateEx 832 28.6560 0.0310 0.0640 0.0344
MXNDArrayWaitAll 712 15443.7207 5.6130 25.0610 21.6906
MXNDArraySyncCopyToCPU 1544 12311.6230 0.0600 24.1980 7.9738
MXNDArrayGetDType 1544 0.5610 0.0000 0.0010 0.0004
MXNDArrayGetShape 3920 1.8360 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1664 14.5700 0.0070 0.0280 0.0088
MXNDArrayFree 18536 190.3200 0.0000 0.9140 0.0103
MXNet C API Calls 82784 570.1410 487.3580 570.1410 41.3915
MXNet C API Concurrency 165567 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1424 0.6730 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7120 3.7110 0.0000 0.0230 0.0005
MXNDArraySetGradState 7120 3.6370 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1424 0.6030 0.0000 0.0010 0.0004
MXAutogradBackwardEx 712 394.2760 0.5250 0.9450 0.5538
MXNDArrayGetContext 11528 5.0250 0.0000 0.0070 0.0004
MXImperativeInvokeEx 23992 613.9530 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2848 6969.1108 0.7010 3.7500 2.4470
FullyConnected 4992 481.0930 0.0080 0.3430 0.0964
argmax 1664 32.9470 0.0050 0.0480 0.0198
_backward_pick 1424 22.6040 0.0130 0.0320 0.0159
_backward_mul_scalar 1424 6.2340 0.0030 0.0220 0.0044
DeleteVariable 61280 439.1900 0.0010 0.9010 0.0072
Flatten 1664 69.9720 0.0080 0.1120 0.0421
Activation 6656 309.0760 0.0030 1.9440 0.0464
SetValueOp 1424 9.2260 0.0050 0.0180 0.0065
pick 1424 11.2390 0.0060 0.0280 0.0079
Pooling 3328 4655.7710 0.0640 3.1720 1.3990
Convolution 3328 18972.7246 0.2260 13.2050 5.7009
CopyCPU2CPU 1664 9.6950 0.0030 0.0360 0.0058
log_softmax 1424 44.3470 0.0150 0.0550 0.0311
_backward_copy 1424 73.5120 0.0220 0.0760 0.0516
_backward_Convolution 2848 26348.9395 2.6240 12.5610 9.2517
_mul_scalar 1424 5.0970 0.0020 0.0230 0.0036
_backward_log_softmax 1424 24.2890 0.0090 0.0400 0.0171
mean 4512 86.3230 0.0080 0.0730 0.0191
_backward_mean 1424 15.6740 0.0080 0.1030 0.0110
_backward_FullyConnected 4272 818.3140 0.0280 0.5940 0.1916
WaitForVar 3088 27.4350 0.0060 0.0280 0.0089
broadcast_equal 1664 8.3610 0.0030 0.0110 0.0050
sgd_update 14240 141.1820 0.0020 0.0790 0.0099
_backward_Activation 5696 420.7540 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56362 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 4998 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 712 2860.2051 0.3060 73.5870 4.0171
MXNDArrayCreateEx 833 28.6890 0.0310 0.0640 0.0344
MXNDArrayWaitAll 713 15465.6094 5.6130 25.0610 21.6909
MXNDArraySyncCopyToCPU 1546 12326.1309 0.0600 24.1980 7.9729
MXNDArrayGetDType 1546 0.5620 0.0000 0.0010 0.0004
MXNDArrayGetShape 3925 1.8390 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1666 14.5880 0.0070 0.0280 0.0088
MXNDArrayFree 18559 190.5540 0.0000 0.9140 0.0103
MXNet C API Calls 82891 570.2480 487.3580 570.2480 41.4450
MXNet C API Concurrency 165781 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1426 0.6740 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7130 3.7150 0.0000 0.0230 0.0005
MXNDArraySetGradState 7130 3.6440 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1426 0.6050 0.0000 0.0010 0.0004
MXAutogradBackwardEx 713 394.8040 0.5250 0.9450 0.5537
MXNDArrayGetContext 11542 5.0290 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24023 614.7190 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2852 6978.9150 0.7010 3.7500 2.4470
FullyConnected 4998 481.6160 0.0080 0.3430 0.0964
argmax 1666 32.9850 0.0050 0.0480 0.0198
_backward_pick 1426 22.6340 0.0130 0.0320 0.0159
_backward_mul_scalar 1426 6.2420 0.0030 0.0220 0.0044
DeleteVariable 61360 439.7610 0.0010 0.9010 0.0072
Flatten 1666 70.0490 0.0080 0.1120 0.0420
Activation 6664 309.4720 0.0030 1.9440 0.0464
SetValueOp 1426 9.2380 0.0050 0.0180 0.0065
pick 1426 11.2540 0.0060 0.0280 0.0079
Pooling 3332 4661.2100 0.0640 3.1720 1.3989
Convolution 3332 18994.8711 0.2260 13.2050 5.7007
CopyCPU2CPU 1666 9.7040 0.0030 0.0360 0.0058
log_softmax 1426 44.4070 0.0150 0.0550 0.0311
_backward_copy 1426 73.6180 0.0220 0.0760 0.0516
_backward_Convolution 2852 26385.9629 2.6240 12.5610 9.2517
_mul_scalar 1426 5.1040 0.0020 0.0230 0.0036
_backward_log_softmax 1426 24.3200 0.0090 0.0400 0.0171
mean 4518 86.4460 0.0080 0.0730 0.0191
_backward_mean 1426 15.6960 0.0080 0.1030 0.0110
_backward_FullyConnected 4278 819.3840 0.0280 0.5940 0.1915
WaitForVar 3092 27.4700 0.0060 0.0280 0.0089
broadcast_equal 1666 8.3710 0.0030 0.0110 0.0050
sgd_update 14260 141.3850 0.0020 0.0790 0.0099
_backward_Activation 5704 421.3370 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56436 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5004 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 713 2864.1189 0.3060 73.5870 4.0170
MXNDArrayCreateEx 834 28.7220 0.0310 0.0640 0.0344
MXNDArrayWaitAll 714 15487.5342 5.6130 25.0610 21.6912
MXNDArraySyncCopyToCPU 1548 12340.5732 0.0600 24.1980 7.9719
MXNDArrayGetDType 1548 0.5640 0.0000 0.0010 0.0004
MXNDArrayGetShape 3930 1.8430 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1668 14.6050 0.0070 0.0280 0.0088
MXNDArrayFree 18582 190.7850 0.0000 0.9140 0.0103
MXNet C API Calls 82998 570.3550 487.3580 570.3550 41.4985
MXNet C API Concurrency 165995 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1428 0.6750 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7140 3.7200 0.0000 0.0230 0.0005
MXNDArraySetGradState 7140 3.6500 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1428 0.6050 0.0000 0.0010 0.0004
MXAutogradBackwardEx 714 395.3570 0.5250 0.9450 0.5537
MXNDArrayGetContext 11556 5.0330 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24054 615.4890 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2856 6988.7661 0.7010 3.7500 2.4470
FullyConnected 5004 482.1440 0.0080 0.3430 0.0964
argmax 1668 33.0240 0.0050 0.0480 0.0198
_backward_pick 1428 22.6650 0.0130 0.0320 0.0159
_backward_mul_scalar 1428 6.2500 0.0030 0.0220 0.0044
DeleteVariable 61440 440.2930 0.0010 0.9010 0.0072
Flatten 1668 70.1290 0.0080 0.1120 0.0420
Activation 6672 309.8440 0.0030 1.9440 0.0464
SetValueOp 1428 9.2520 0.0050 0.0180 0.0065
pick 1428 11.2690 0.0060 0.0280 0.0079
Pooling 3336 4666.6030 0.0640 3.1720 1.3989
Convolution 3336 19017.1152 0.2260 13.2050 5.7006
CopyCPU2CPU 1668 9.7130 0.0030 0.0360 0.0058
log_softmax 1428 44.4660 0.0150 0.0550 0.0311
_backward_copy 1428 73.7240 0.0220 0.0760 0.0516
_backward_Convolution 2856 26423.0039 2.6240 12.5610 9.2518
_mul_scalar 1428 5.1110 0.0020 0.0230 0.0036
_backward_log_softmax 1428 24.3540 0.0090 0.0400 0.0171
mean 4524 86.5650 0.0080 0.0730 0.0191
_backward_mean 1428 15.7170 0.0080 0.1030 0.0110
_backward_FullyConnected 4284 820.4540 0.0280 0.5940 0.1915
WaitForVar 3096 27.5050 0.0060 0.0280 0.0089
broadcast_equal 1668 8.3810 0.0030 0.0110 0.0050
sgd_update 14280 141.5380 0.0020 0.0790 0.0099
_backward_Activation 5712 421.9710 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56510 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5010 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 714 2868.0271 0.3060 73.5870 4.0168
MXNDArrayCreateEx 835 28.7540 0.0310 0.0640 0.0344
MXNDArrayWaitAll 715 15509.2217 5.6130 25.0610 21.6912
MXNDArraySyncCopyToCPU 1550 12354.9580 0.0600 24.1980 7.9709
MXNDArrayGetDType 1550 0.5660 0.0000 0.0010 0.0004
MXNDArrayGetShape 3935 1.8440 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1670 14.6220 0.0070 0.0280 0.0088
MXNDArrayFree 18605 191.0060 0.0000 0.9140 0.0103
MXNet C API Calls 83105 570.4620 487.3580 570.4620 41.5520
MXNet C API Concurrency 166209 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1430 0.6750 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7150 3.7260 0.0000 0.0230 0.0005
MXNDArraySetGradState 7150 3.6530 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1430 0.6050 0.0000 0.0010 0.0004
MXAutogradBackwardEx 715 395.8990 0.5250 0.9450 0.5537
MXNDArrayGetContext 11570 5.0410 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24085 616.2870 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2860 6998.4419 0.7010 3.7500 2.4470
FullyConnected 5010 482.6760 0.0080 0.3430 0.0963
argmax 1670 33.0620 0.0050 0.0480 0.0198
_backward_pick 1430 22.6970 0.0130 0.0320 0.0159
_backward_mul_scalar 1430 6.2590 0.0030 0.0220 0.0044
DeleteVariable 61520 440.8150 0.0010 0.9010 0.0072
Flatten 1670 70.2090 0.0080 0.1120 0.0420
Activation 6680 310.2130 0.0030 1.9440 0.0464
SetValueOp 1430 9.2650 0.0050 0.0180 0.0065
pick 1430 11.2840 0.0060 0.0280 0.0079
Pooling 3340 4671.9932 0.0640 3.1720 1.3988
Convolution 3340 19039.3340 0.2260 13.2050 5.7004
CopyCPU2CPU 1670 9.7230 0.0030 0.0360 0.0058
log_softmax 1430 44.5240 0.0150 0.0550 0.0311
_backward_copy 1430 73.8300 0.0220 0.0760 0.0516
_backward_Convolution 2860 26459.7754 2.6240 12.5610 9.2517
_mul_scalar 1430 5.1170 0.0020 0.0230 0.0036
_backward_log_softmax 1430 24.3860 0.0090 0.0400 0.0171
mean 4530 86.6820 0.0080 0.0730 0.0191
_backward_mean 1430 15.7380 0.0080 0.1030 0.0110
_backward_FullyConnected 4290 821.5250 0.0280 0.5940 0.1915
WaitForVar 3100 27.5420 0.0060 0.0280 0.0089
broadcast_equal 1670 8.3910 0.0030 0.0110 0.0050
sgd_update 14300 141.6890 0.0020 0.0790 0.0099
_backward_Activation 5720 422.6060 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56584 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5016 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 715 2871.9290 0.3060 73.5870 4.0167
MXNDArrayCreateEx 836 28.7870 0.0310 0.0640 0.0344
MXNDArrayWaitAll 716 15530.9287 5.6130 25.0610 21.6912
MXNDArraySyncCopyToCPU 1552 12369.4805 0.0600 24.1980 7.9700
MXNDArrayGetDType 1552 0.5670 0.0000 0.0010 0.0004
MXNDArrayGetShape 3940 1.8470 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1672 14.6390 0.0070 0.0280 0.0088
MXNDArrayFree 18628 191.2320 0.0000 0.9140 0.0103
MXNet C API Calls 83212 570.5690 487.3580 570.5690 41.6055
MXNet C API Concurrency 166423 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1432 0.6770 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7160 3.7320 0.0000 0.0230 0.0005
MXNDArraySetGradState 7160 3.6580 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1432 0.6060 0.0000 0.0010 0.0004
MXAutogradBackwardEx 716 396.4320 0.5250 0.9450 0.5537
MXNDArrayGetContext 11584 5.0490 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24116 617.0680 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2864 7008.0801 0.7010 3.7500 2.4470
FullyConnected 5016 483.2900 0.0080 0.3430 0.0963
argmax 1672 33.1000 0.0050 0.0480 0.0198
_backward_pick 1432 22.7300 0.0130 0.0320 0.0159
_backward_mul_scalar 1432 6.2670 0.0030 0.0220 0.0044
DeleteVariable 61600 441.3530 0.0010 0.9010 0.0072
Flatten 1672 70.2880 0.0080 0.1120 0.0420
Activation 6688 310.5800 0.0030 1.9440 0.0464
SetValueOp 1432 9.2790 0.0050 0.0180 0.0065
pick 1432 11.2990 0.0060 0.0280 0.0079
Pooling 3344 4677.3618 0.0640 3.1720 1.3987
Convolution 3344 19061.4629 0.2260 13.2050 5.7002
CopyCPU2CPU 1672 9.7330 0.0030 0.0360 0.0058
log_softmax 1432 44.5870 0.0150 0.0550 0.0311
_backward_copy 1432 73.9340 0.0220 0.0760 0.0516
_backward_Convolution 2864 26496.5898 2.6240 12.5610 9.2516
_mul_scalar 1432 5.1240 0.0020 0.0230 0.0036
_backward_log_softmax 1432 24.4220 0.0090 0.0400 0.0171
mean 4536 86.7940 0.0080 0.0730 0.0191
_backward_mean 1432 15.7600 0.0080 0.1030 0.0110
_backward_FullyConnected 4296 822.7220 0.0280 0.5940 0.1915
WaitForVar 3104 27.5770 0.0060 0.0280 0.0089
broadcast_equal 1672 8.4010 0.0030 0.0110 0.0050
sgd_update 14320 141.9310 0.0020 0.0790 0.0099
_backward_Activation 5728 423.1870 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56658 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5022 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 716 2875.8259 0.3060 73.5870 4.0165
MXNDArrayCreateEx 837 28.8210 0.0310 0.0640 0.0344
MXNDArrayWaitAll 717 15552.7402 5.6130 25.0610 21.6914
MXNDArraySyncCopyToCPU 1554 12383.9414 0.0600 24.1980 7.9691
MXNDArrayGetDType 1554 0.5680 0.0000 0.0010 0.0004
MXNDArrayGetShape 3945 1.8490 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1674 14.6560 0.0070 0.0280 0.0088
MXNDArrayFree 18651 191.4680 0.0000 0.9140 0.0103
MXNet C API Calls 83319 570.6760 487.3580 570.6760 41.6590
MXNet C API Concurrency 166637 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1434 0.6780 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7170 3.7340 0.0000 0.0230 0.0005
MXNDArraySetGradState 7170 3.6620 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1434 0.6080 0.0000 0.0010 0.0004
MXAutogradBackwardEx 717 396.9610 0.5250 0.9450 0.5536
MXNDArrayGetContext 11598 5.0530 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24147 617.8380 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2868 7017.8018 0.7010 3.7500 2.4469
FullyConnected 5022 483.8180 0.0080 0.3430 0.0963
argmax 1674 33.1390 0.0050 0.0480 0.0198
_backward_pick 1434 22.7620 0.0130 0.0320 0.0159
_backward_mul_scalar 1434 6.2760 0.0030 0.0220 0.0044
DeleteVariable 61680 441.9120 0.0010 0.9010 0.0072
Flatten 1674 70.3680 0.0080 0.1120 0.0420
Activation 6696 310.9680 0.0030 1.9440 0.0464
SetValueOp 1434 9.2920 0.0050 0.0180 0.0065
pick 1434 11.3140 0.0060 0.0280 0.0079
Pooling 3348 4682.7358 0.0640 3.1720 1.3987
Convolution 3348 19083.6211 0.2260 13.2050 5.7000
CopyCPU2CPU 1674 9.7430 0.0030 0.0360 0.0058
log_softmax 1434 44.6460 0.0150 0.0550 0.0311
_backward_copy 1434 74.0400 0.0220 0.0760 0.0516
_backward_Convolution 2868 26533.5957 2.6240 12.5610 9.2516
_mul_scalar 1434 5.1310 0.0020 0.0230 0.0036
_backward_log_softmax 1434 24.4540 0.0090 0.0400 0.0171
mean 4542 86.9150 0.0080 0.0730 0.0191
_backward_mean 1434 15.7820 0.0080 0.1030 0.0110
_backward_FullyConnected 4302 823.7920 0.0280 0.5940 0.1915
WaitForVar 3108 27.6120 0.0060 0.0280 0.0089
broadcast_equal 1674 8.4110 0.0030 0.0110 0.0050
sgd_update 14340 142.1400 0.0020 0.0790 0.0099
_backward_Activation 5736 423.7790 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56732 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5028 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 717 2879.7310 0.3060 73.5870 4.0164
MXNDArrayCreateEx 838 28.8550 0.0310 0.0640 0.0344
MXNDArrayWaitAll 718 15574.6299 5.6130 25.0610 21.6917
MXNDArraySyncCopyToCPU 1556 12398.4443 0.0600 24.1980 7.9682
MXNDArrayGetDType 1556 0.5680 0.0000 0.0010 0.0004
MXNDArrayGetShape 3950 1.8520 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1676 14.6730 0.0070 0.0280 0.0088
MXNDArrayFree 18674 191.6960 0.0000 0.9140 0.0103
MXNet C API Calls 83426 570.7830 487.3580 570.7830 41.7125
MXNet C API Concurrency 166851 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1436 0.6790 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7180 3.7380 0.0000 0.0230 0.0005
MXNDArraySetGradState 7180 3.6680 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1436 0.6090 0.0000 0.0010 0.0004
MXAutogradBackwardEx 718 397.4890 0.5250 0.9450 0.5536
MXNDArrayGetContext 11612 5.0580 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24178 618.6040 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2872 7027.5181 0.7010 3.7500 2.4469
FullyConnected 5028 484.3640 0.0080 0.3430 0.0963
argmax 1676 33.1780 0.0050 0.0480 0.0198
_backward_pick 1436 22.7930 0.0130 0.0320 0.0159
_backward_mul_scalar 1436 6.2860 0.0030 0.0220 0.0044
DeleteVariable 61760 442.4500 0.0010 0.9010 0.0072
Flatten 1676 70.4450 0.0080 0.1120 0.0420
Activation 6704 311.3350 0.0030 1.9440 0.0464
SetValueOp 1436 9.3050 0.0050 0.0180 0.0065
pick 1436 11.3300 0.0060 0.0280 0.0079
Pooling 3352 4688.0649 0.0640 3.1720 1.3986
Convolution 3352 19105.9043 0.2260 13.2050 5.6999
CopyCPU2CPU 1676 9.7530 0.0030 0.0360 0.0058
log_softmax 1436 44.7050 0.0150 0.0550 0.0311
_backward_copy 1436 74.1440 0.0220 0.0760 0.0516
_backward_Convolution 2872 26570.7109 2.6240 12.5610 9.2516
_mul_scalar 1436 5.1380 0.0020 0.0230 0.0036
_backward_log_softmax 1436 24.4860 0.0090 0.0400 0.0171
mean 4548 87.0310 0.0080 0.0730 0.0191
_backward_mean 1436 15.8040 0.0080 0.1030 0.0110
_backward_FullyConnected 4308 824.8680 0.0280 0.5940 0.1915
WaitForVar 3112 27.6490 0.0060 0.0280 0.0089
broadcast_equal 1676 8.4200 0.0030 0.0110 0.0050
sgd_update 14360 142.3880 0.0020 0.0790 0.0099
_backward_Activation 5744 424.3580 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56806 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5034 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 718 2883.7141 0.3060 73.5870 4.0163
MXNDArrayCreateEx 839 28.8900 0.0310 0.0640 0.0344
MXNDArrayWaitAll 719 15596.3457 5.6130 25.0610 21.6917
MXNDArraySyncCopyToCPU 1558 12412.6709 0.0600 24.1980 7.9671
MXNDArrayGetDType 1558 0.5690 0.0000 0.0010 0.0004
MXNDArrayGetShape 3955 1.8540 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1678 14.6920 0.0070 0.0280 0.0088
MXNDArrayFree 18697 191.9340 0.0000 0.9140 0.0103
MXNet C API Calls 83533 570.8900 487.3580 570.8900 41.7660
MXNet C API Concurrency 167065 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1438 0.6810 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7190 3.7410 0.0000 0.0230 0.0005
MXNDArraySetGradState 7190 3.6730 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1438 0.6090 0.0000 0.0010 0.0004
MXAutogradBackwardEx 719 398.0440 0.5250 0.9450 0.5536
MXNDArrayGetContext 11626 5.0640 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24209 619.4640 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2876 7037.1880 0.7010 3.7500 2.4469
FullyConnected 5034 484.8910 0.0080 0.3430 0.0963
argmax 1678 33.2150 0.0050 0.0480 0.0198
_backward_pick 1438 22.8250 0.0130 0.0320 0.0159
_backward_mul_scalar 1438 6.2950 0.0030 0.0220 0.0044
DeleteVariable 61840 442.9820 0.0010 0.9010 0.0072
Flatten 1678 70.5250 0.0080 0.1120 0.0420
Activation 6712 311.7020 0.0030 1.9440 0.0464
SetValueOp 1438 9.3180 0.0050 0.0180 0.0065
pick 1438 11.3450 0.0060 0.0280 0.0079
Pooling 3356 4693.4780 0.0640 3.1720 1.3985
Convolution 3356 19128.0879 0.2260 13.2050 5.6997
CopyCPU2CPU 1678 9.7620 0.0030 0.0360 0.0058
log_softmax 1438 44.7640 0.0150 0.0550 0.0311
_backward_copy 1438 74.2480 0.0220 0.0760 0.0516
_backward_Convolution 2876 26607.5254 2.6240 12.5610 9.2516
_mul_scalar 1438 5.1460 0.0020 0.0230 0.0036
_backward_log_softmax 1438 24.5180 0.0090 0.0400 0.0171
mean 4554 87.1470 0.0080 0.0730 0.0191
_backward_mean 1438 15.8250 0.0080 0.1030 0.0110
_backward_FullyConnected 4314 825.9380 0.0280 0.5940 0.1915
WaitForVar 3116 27.6850 0.0060 0.0280 0.0089
broadcast_equal 1678 8.4300 0.0030 0.0110 0.0050
sgd_update 14380 142.5420 0.0020 0.0790 0.0099
_backward_Activation 5752 425.0200 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56880 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5040 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 719 2887.6321 0.3060 73.5870 4.0162
MXNDArrayCreateEx 840 28.9260 0.0310 0.0640 0.0344
MXNDArrayWaitAll 720 15617.9961 5.6130 25.0610 21.6917
MXNDArraySyncCopyToCPU 1560 12427.0449 0.0600 24.1980 7.9661
MXNDArrayGetDType 1560 0.5690 0.0000 0.0010 0.0004
MXNDArrayGetShape 3960 1.8570 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1680 14.7110 0.0070 0.0280 0.0088
MXNDArrayFree 18720 192.1960 0.0000 0.9140 0.0103
MXNet C API Calls 83640 570.9970 487.3580 570.9970 41.8195
MXNet C API Concurrency 167279 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1440 0.6820 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7200 3.7460 0.0000 0.0230 0.0005
MXNDArraySetGradState 7200 3.6780 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1440 0.6090 0.0000 0.0010 0.0004
MXAutogradBackwardEx 720 398.5900 0.5250 0.9450 0.5536
MXNDArrayGetContext 11640 5.0690 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24240 620.2880 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2880 7046.9380 0.7010 3.7500 2.4469
FullyConnected 5040 485.5130 0.0080 0.3430 0.0963
argmax 1680 33.2520 0.0050 0.0480 0.0198
_backward_pick 1440 22.8550 0.0130 0.0320 0.0159
_backward_mul_scalar 1440 6.3040 0.0030 0.0220 0.0044
DeleteVariable 61920 443.5560 0.0010 0.9010 0.0072
Flatten 1680 70.6050 0.0080 0.1120 0.0420
Activation 6720 312.0900 0.0030 1.9440 0.0464
SetValueOp 1440 9.3310 0.0050 0.0180 0.0065
pick 1440 11.3600 0.0060 0.0280 0.0079
Pooling 3360 4698.8398 0.0640 3.1720 1.3985
Convolution 3360 19150.2070 0.2260 13.2050 5.6995
CopyCPU2CPU 1680 9.7710 0.0030 0.0360 0.0058
log_softmax 1440 44.8260 0.0150 0.0550 0.0311
_backward_copy 1440 74.3530 0.0220 0.0760 0.0516
_backward_Convolution 2880 26644.2031 2.6240 12.5610 9.2515
_mul_scalar 1440 5.1530 0.0020 0.0230 0.0036
_backward_log_softmax 1440 24.5600 0.0090 0.0400 0.0171
mean 4560 87.2560 0.0080 0.0730 0.0191
_backward_mean 1440 15.8460 0.0080 0.1030 0.0110
_backward_FullyConnected 4320 827.1010 0.0280 0.5940 0.1915
WaitForVar 3120 27.7190 0.0060 0.0280 0.0089
broadcast_equal 1680 8.4390 0.0030 0.0110 0.0050
sgd_update 14400 142.7810 0.0020 0.0790 0.0099
_backward_Activation 5760 425.6100 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 56954 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5046 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 720 2891.5559 0.3060 73.5870 4.0160
MXNDArrayCreateEx 841 28.9620 0.0310 0.0640 0.0344
MXNDArrayWaitAll 721 15639.8428 5.6130 25.0610 21.6919
MXNDArraySyncCopyToCPU 1562 12441.1201 0.0600 24.1980 7.9649
MXNDArrayGetDType 1562 0.5690 0.0000 0.0010 0.0004
MXNDArrayGetShape 3965 1.8590 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1682 14.7290 0.0070 0.0280 0.0088
MXNDArrayFree 18743 192.4320 0.0000 0.9140 0.0103
MXNet C API Calls 83747 571.1040 487.3580 571.1040 41.8730
MXNet C API Concurrency 167493 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1442 0.6820 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7210 3.7520 0.0000 0.0230 0.0005
MXNDArraySetGradState 7210 3.6810 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1442 0.6110 0.0000 0.0010 0.0004
MXAutogradBackwardEx 721 399.1920 0.5250 0.9450 0.5537
MXNDArrayGetContext 11654 5.0790 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24271 621.1360 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2884 7056.6201 0.7010 3.7500 2.4468
FullyConnected 5046 486.0490 0.0080 0.3430 0.0963
argmax 1682 33.2900 0.0050 0.0480 0.0198
_backward_pick 1442 22.8870 0.0130 0.0320 0.0159
_backward_mul_scalar 1442 6.3130 0.0030 0.0220 0.0044
DeleteVariable 62000 444.0990 0.0010 0.9010 0.0072
Flatten 1682 70.6820 0.0080 0.1120 0.0420
Activation 6728 312.4610 0.0030 1.9440 0.0464
SetValueOp 1442 9.3440 0.0050 0.0180 0.0065
pick 1442 11.3750 0.0060 0.0280 0.0079
Pooling 3364 4704.2412 0.0640 3.1720 1.3984
Convolution 3364 19172.3867 0.2260 13.2050 5.6993
CopyCPU2CPU 1682 9.7800 0.0030 0.0360 0.0058
log_softmax 1442 44.8850 0.0150 0.0550 0.0311
_backward_copy 1442 74.4580 0.0220 0.0760 0.0516
_backward_Convolution 2884 26681.2402 2.6240 12.5610 9.2515
_mul_scalar 1442 5.1610 0.0020 0.0230 0.0036
_backward_log_softmax 1442 24.5930 0.0090 0.0400 0.0171
mean 4566 87.3740 0.0080 0.0730 0.0191
_backward_mean 1442 15.8670 0.0080 0.1030 0.0110
_backward_FullyConnected 4326 828.1770 0.0280 0.5940 0.1914
WaitForVar 3124 27.7540 0.0060 0.0280 0.0089
broadcast_equal 1682 8.4480 0.0030 0.0110 0.0050
sgd_update 14420 142.9310 0.0020 0.0790 0.0099
_backward_Activation 5768 426.2550 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57028 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5052 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 721 2895.4741 0.3060 73.5870 4.0159
MXNDArrayCreateEx 842 28.9980 0.0310 0.0640 0.0344
MXNDArrayWaitAll 722 15661.7002 5.6130 25.0610 21.6921
MXNDArraySyncCopyToCPU 1564 12455.4023 0.0600 24.1980 7.9638
MXNDArrayGetDType 1564 0.5690 0.0000 0.0010 0.0004
MXNDArrayGetShape 3970 1.8600 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1684 14.7480 0.0070 0.0280 0.0088
MXNDArrayFree 18766 192.6680 0.0000 0.9140 0.0103
MXNet C API Calls 83854 571.2110 487.3580 571.2110 41.9265
MXNet C API Concurrency 167707 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1444 0.6830 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7220 3.7560 0.0000 0.0230 0.0005
MXNDArraySetGradState 7220 3.6840 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1444 0.6120 0.0000 0.0010 0.0004
MXAutogradBackwardEx 722 399.7480 0.5250 0.9450 0.5537
MXNDArrayGetContext 11668 5.0860 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24302 621.9680 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2888 7066.3311 0.7010 3.7500 2.4468
FullyConnected 5052 486.5710 0.0080 0.3430 0.0963
argmax 1684 33.3290 0.0050 0.0480 0.0198
_backward_pick 1444 22.9170 0.0130 0.0320 0.0159
_backward_mul_scalar 1444 6.3220 0.0030 0.0220 0.0044
DeleteVariable 62080 444.6360 0.0010 0.9010 0.0072
Flatten 1684 70.7590 0.0080 0.1120 0.0420
Activation 6736 312.8290 0.0030 1.9440 0.0464
SetValueOp 1444 9.3570 0.0050 0.0180 0.0065
pick 1444 11.3900 0.0060 0.0280 0.0079
Pooling 3368 4709.6279 0.0640 3.1720 1.3983
Convolution 3368 19194.6211 0.2260 13.2050 5.6991
CopyCPU2CPU 1684 9.7900 0.0030 0.0360 0.0058
log_softmax 1444 44.9450 0.0150 0.0550 0.0311
_backward_copy 1444 74.5650 0.0220 0.0760 0.0516
_backward_Convolution 2888 26718.3496 2.6240 12.5610 9.2515
_mul_scalar 1444 5.1690 0.0020 0.0230 0.0036
_backward_log_softmax 1444 24.6260 0.0090 0.0400 0.0171
mean 4572 87.4900 0.0080 0.0730 0.0191
_backward_mean 1444 15.8890 0.0080 0.1030 0.0110
_backward_FullyConnected 4332 829.2500 0.0280 0.5940 0.1914
WaitForVar 3128 27.7890 0.0060 0.0280 0.0089
broadcast_equal 1684 8.4570 0.0030 0.0110 0.0050
sgd_update 14440 143.0840 0.0020 0.0790 0.0099
_backward_Activation 5776 426.9060 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57102 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5058 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 722 2899.3960 0.3060 73.5870 4.0158
MXNDArrayCreateEx 843 29.0320 0.0310 0.0640 0.0344
MXNDArrayWaitAll 723 15683.4092 5.6130 25.0610 21.6921
MXNDArraySyncCopyToCPU 1566 12469.5059 0.0600 24.1980 7.9626
MXNDArrayGetDType 1566 0.5700 0.0000 0.0010 0.0004
MXNDArrayGetShape 3975 1.8610 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1686 14.7660 0.0070 0.0280 0.0088
MXNDArrayFree 18789 192.9060 0.0000 0.9140 0.0103
MXNet C API Calls 83961 571.3180 487.3580 571.3180 41.9800
MXNet C API Concurrency 167921 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1446 0.6830 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7230 3.7630 0.0000 0.0230 0.0005
MXNDArraySetGradState 7230 3.6910 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1446 0.6130 0.0000 0.0010 0.0004
MXAutogradBackwardEx 723 400.3200 0.5250 0.9450 0.5537
MXNDArrayGetContext 11682 5.0940 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24333 622.8190 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2892 7076.0708 0.7010 3.7500 2.4468
FullyConnected 5058 487.1370 0.0080 0.3430 0.0963
argmax 1686 33.3680 0.0050 0.0480 0.0198
_backward_pick 1446 22.9480 0.0130 0.0320 0.0159
_backward_mul_scalar 1446 6.3310 0.0030 0.0220 0.0044
DeleteVariable 62160 445.1740 0.0010 0.9010 0.0072
Flatten 1686 70.8390 0.0080 0.1120 0.0420
Activation 6744 313.1990 0.0030 1.9440 0.0464
SetValueOp 1446 9.3710 0.0050 0.0180 0.0065
pick 1446 11.4050 0.0060 0.0280 0.0079
Pooling 3372 4714.9619 0.0640 3.1720 1.3983
Convolution 3372 19216.8301 0.2260 13.2050 5.6989
CopyCPU2CPU 1686 9.8000 0.0030 0.0360 0.0058
log_softmax 1446 45.0060 0.0150 0.0550 0.0311
_backward_copy 1446 74.6700 0.0220 0.0760 0.0516
_backward_Convolution 2892 26755.1230 2.6240 12.5610 9.2514
_mul_scalar 1446 5.1760 0.0020 0.0230 0.0036
_backward_log_softmax 1446 24.6590 0.0090 0.0400 0.0171
mean 4578 87.6080 0.0080 0.0730 0.0191
_backward_mean 1446 15.9110 0.0080 0.1030 0.0110
_backward_FullyConnected 4338 830.3200 0.0280 0.5940 0.1914
WaitForVar 3132 27.8240 0.0060 0.0280 0.0089
broadcast_equal 1686 8.4670 0.0030 0.0110 0.0050
sgd_update 14460 143.2910 0.0020 0.0790 0.0099
_backward_Activation 5784 427.4740 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57176 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5064 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 723 2903.3291 0.3060 73.5870 4.0157
MXNDArrayCreateEx 844 29.0680 0.0310 0.0640 0.0344
MXNDArrayWaitAll 724 15705.0693 5.6130 25.0610 21.6921
MXNDArraySyncCopyToCPU 1568 12484.1025 0.0600 24.1980 7.9618
MXNDArrayGetDType 1568 0.5700 0.0000 0.0010 0.0004
MXNDArrayGetShape 3980 1.8630 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1688 14.7840 0.0070 0.0280 0.0088
MXNDArrayFree 18812 193.1580 0.0000 0.9140 0.0103
MXNet C API Calls 84068 571.4250 487.3580 571.4250 42.0335
MXNet C API Concurrency 168135 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1448 0.6840 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7240 3.7680 0.0000 0.0230 0.0005
MXNDArraySetGradState 7240 3.6950 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1448 0.6130 0.0000 0.0010 0.0004
MXAutogradBackwardEx 724 400.8680 0.5250 0.9450 0.5537
MXNDArrayGetContext 11696 5.1010 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24364 623.6700 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2896 7085.7510 0.7010 3.7500 2.4467
FullyConnected 5064 487.8660 0.0080 0.3430 0.0963
argmax 1688 33.4120 0.0050 0.0480 0.0198
_backward_pick 1448 22.9800 0.0130 0.0320 0.0159
_backward_mul_scalar 1448 6.3390 0.0030 0.0220 0.0044
DeleteVariable 62240 445.7630 0.0010 0.9010 0.0072
Flatten 1688 70.9210 0.0080 0.1120 0.0420
Activation 6752 313.5830 0.0030 1.9440 0.0464
SetValueOp 1448 9.3840 0.0050 0.0180 0.0065
pick 1448 11.4200 0.0060 0.0280 0.0079
Pooling 3376 4720.3618 0.0640 3.1720 1.3982
Convolution 3376 19239.0352 0.2260 13.2050 5.6988
CopyCPU2CPU 1688 9.8090 0.0030 0.0360 0.0058
log_softmax 1448 45.0770 0.0150 0.0550 0.0311
_backward_copy 1448 74.7730 0.0220 0.0760 0.0516
_backward_Convolution 2896 26791.8574 2.6240 12.5610 9.2513
_mul_scalar 1448 5.1840 0.0020 0.0230 0.0036
_backward_log_softmax 1448 24.7010 0.0090 0.0400 0.0171
mean 4584 87.7160 0.0080 0.0730 0.0191
_backward_mean 1448 15.9330 0.0080 0.1030 0.0110
_backward_FullyConnected 4344 831.7250 0.0280 0.5940 0.1915
WaitForVar 3136 27.8600 0.0060 0.0280 0.0089
broadcast_equal 1688 8.4770 0.0030 0.0110 0.0050
sgd_update 14480 143.5750 0.0020 0.0790 0.0099
_backward_Activation 5792 428.0860 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57250 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5070 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 724 2907.2590 0.3060 73.5870 4.0156
MXNDArrayCreateEx 845 29.1030 0.0310 0.0640 0.0344
MXNDArrayWaitAll 725 15726.9629 5.6130 25.0610 21.6924
MXNDArraySyncCopyToCPU 1570 12498.1641 0.0600 24.1980 7.9606
MXNDArrayGetDType 1570 0.5700 0.0000 0.0010 0.0004
MXNDArrayGetShape 3985 1.8650 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1690 14.8020 0.0070 0.0280 0.0088
MXNDArrayFree 18835 193.3950 0.0000 0.9140 0.0103
MXNet C API Calls 84175 571.5320 487.3580 571.5320 42.0870
MXNet C API Concurrency 168349 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1450 0.6840 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7250 3.7720 0.0000 0.0230 0.0005
MXNDArraySetGradState 7250 3.7000 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1450 0.6150 0.0000 0.0010 0.0004
MXAutogradBackwardEx 725 401.4370 0.5250 0.9450 0.5537
MXNDArrayGetContext 11710 5.1070 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24395 624.5510 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2900 7095.4712 0.7010 3.7500 2.4467
FullyConnected 5070 488.3960 0.0080 0.3430 0.0963
argmax 1690 33.4500 0.0050 0.0480 0.0198
_backward_pick 1450 23.0120 0.0130 0.0320 0.0159
_backward_mul_scalar 1450 6.3480 0.0030 0.0220 0.0044
DeleteVariable 62320 446.2930 0.0010 0.9010 0.0072
Flatten 1690 70.9990 0.0080 0.1120 0.0420
Activation 6760 313.9550 0.0030 1.9440 0.0464
SetValueOp 1450 9.3970 0.0050 0.0180 0.0065
pick 1450 11.4350 0.0060 0.0280 0.0079
Pooling 3380 4725.7158 0.0640 3.1720 1.3981
Convolution 3380 19261.2129 0.2260 13.2050 5.6986
CopyCPU2CPU 1690 9.8190 0.0030 0.0360 0.0058
log_softmax 1450 45.1360 0.0150 0.0550 0.0311
_backward_copy 1450 74.8780 0.0220 0.0760 0.0516
_backward_Convolution 2900 26828.9785 2.6240 12.5610 9.2514
_mul_scalar 1450 5.1910 0.0020 0.0230 0.0036
_backward_log_softmax 1450 24.7330 0.0090 0.0400 0.0171
mean 4590 87.8350 0.0080 0.0730 0.0191
_backward_mean 1450 15.9550 0.0080 0.1030 0.0110
_backward_FullyConnected 4350 832.8320 0.0280 0.5940 0.1915
WaitForVar 3140 27.8960 0.0060 0.0280 0.0089
broadcast_equal 1690 8.4860 0.0030 0.0110 0.0050
sgd_update 14500 143.7780 0.0020 0.0790 0.0099
_backward_Activation 5800 428.6280 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57324 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5076 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 725 2911.1809 0.3060 73.5870 4.0154
MXNDArrayCreateEx 846 29.1400 0.0310 0.0640 0.0344
MXNDArrayWaitAll 726 15748.8643 5.6130 25.0610 21.6926
MXNDArraySyncCopyToCPU 1572 12512.4512 0.0600 24.1980 7.9596
MXNDArrayGetDType 1572 0.5700 0.0000 0.0010 0.0004
MXNDArrayGetShape 3990 1.8680 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1692 14.8200 0.0070 0.0280 0.0088
MXNDArrayFree 18858 193.6370 0.0000 0.9140 0.0103
MXNet C API Calls 84282 571.6390 487.3580 571.6390 42.1405
MXNet C API Concurrency 168563 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1452 0.6840 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7260 3.7770 0.0000 0.0230 0.0005
MXNDArraySetGradState 7260 3.7070 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1452 0.6170 0.0000 0.0010 0.0004
MXAutogradBackwardEx 726 401.9830 0.5250 0.9450 0.5537
MXNDArrayGetContext 11724 5.1120 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24426 625.3950 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2904 7105.2788 0.7010 3.7500 2.4467
FullyConnected 5076 488.9240 0.0080 0.3430 0.0963
argmax 1692 33.4880 0.0050 0.0480 0.0198
_backward_pick 1452 23.0430 0.0130 0.0320 0.0159
_backward_mul_scalar 1452 6.3570 0.0030 0.0220 0.0044
DeleteVariable 62400 446.8400 0.0010 0.9010 0.0072
Flatten 1692 71.0780 0.0080 0.1120 0.0420
Activation 6768 314.3250 0.0030 1.9440 0.0464
SetValueOp 1452 9.4100 0.0050 0.0180 0.0065
pick 1452 11.4510 0.0060 0.0280 0.0079
Pooling 3384 4731.0669 0.0640 3.1720 1.3981
Convolution 3384 19283.4707 0.2260 13.2050 5.6984
CopyCPU2CPU 1692 9.8300 0.0030 0.0360 0.0058
log_softmax 1452 45.1950 0.0150 0.0550 0.0311
_backward_copy 1452 74.9830 0.0220 0.0760 0.0516
_backward_Convolution 2904 26866.0527 2.6240 12.5610 9.2514
_mul_scalar 1452 5.1980 0.0020 0.0230 0.0036
_backward_log_softmax 1452 24.7660 0.0090 0.0400 0.0171
mean 4596 87.9550 0.0080 0.0730 0.0191
_backward_mean 1452 15.9770 0.0080 0.1030 0.0110
_backward_FullyConnected 4356 833.9050 0.0280 0.5940 0.1914
WaitForVar 3144 27.9310 0.0060 0.0280 0.0089
broadcast_equal 1692 8.4950 0.0030 0.0110 0.0050
sgd_update 14520 143.9850 0.0020 0.0790 0.0099
_backward_Activation 5808 429.2110 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57398 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5082 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 726 2915.1050 0.3060 73.5870 4.0153
MXNDArrayCreateEx 847 29.1760 0.0310 0.0640 0.0344
MXNDArrayWaitAll 727 15770.6260 5.6130 25.0610 21.6927
MXNDArraySyncCopyToCPU 1574 12526.6904 0.0600 24.1980 7.9585
MXNDArrayGetDType 1574 0.5710 0.0000 0.0010 0.0004
MXNDArrayGetShape 3995 1.8720 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1694 14.8390 0.0070 0.0280 0.0088
MXNDArrayFree 18881 193.8710 0.0000 0.9140 0.0103
MXNet C API Calls 84389 571.7460 487.3580 571.7460 42.1940
MXNet C API Concurrency 168777 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1454 0.6850 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7270 3.7830 0.0000 0.0230 0.0005
MXNDArraySetGradState 7270 3.7130 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1454 0.6190 0.0000 0.0010 0.0004
MXAutogradBackwardEx 727 402.5300 0.5250 0.9450 0.5537
MXNDArrayGetContext 11738 5.1140 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24457 626.2150 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2908 7115.0562 0.7010 3.7500 2.4467
FullyConnected 5082 489.4520 0.0080 0.3430 0.0963
argmax 1694 33.5270 0.0050 0.0480 0.0198
_backward_pick 1454 23.0740 0.0130 0.0320 0.0159
_backward_mul_scalar 1454 6.3660 0.0030 0.0220 0.0044
DeleteVariable 62480 447.3700 0.0010 0.9010 0.0072
Flatten 1694 71.1570 0.0080 0.1120 0.0420
Activation 6776 314.6960 0.0030 1.9440 0.0464
SetValueOp 1454 9.4230 0.0050 0.0180 0.0065
pick 1454 11.4660 0.0060 0.0280 0.0079
Pooling 3388 4736.4131 0.0640 3.1720 1.3980
Convolution 3388 19305.6797 0.2260 13.2050 5.6983
CopyCPU2CPU 1694 9.8390 0.0030 0.0360 0.0058
log_softmax 1454 45.2540 0.0150 0.0550 0.0311
_backward_copy 1454 75.0890 0.0220 0.0760 0.0516
_backward_Convolution 2908 26902.8594 2.6240 12.5610 9.2513
_mul_scalar 1454 5.2050 0.0020 0.0230 0.0036
_backward_log_softmax 1454 24.7970 0.0090 0.0400 0.0171
mean 4602 88.0710 0.0080 0.0730 0.0191
_backward_mean 1454 15.9990 0.0080 0.1030 0.0110
_backward_FullyConnected 4362 835.0140 0.0280 0.5940 0.1914
WaitForVar 3148 27.9660 0.0060 0.0280 0.0089
broadcast_equal 1694 8.5040 0.0030 0.0110 0.0050
sgd_update 14540 144.1880 0.0020 0.0790 0.0099
_backward_Activation 5816 429.7780 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57472 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5088 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 727 2919.0391 0.3060 73.5870 4.0152
MXNDArrayCreateEx 848 29.2130 0.0310 0.0640 0.0344
MXNDArrayWaitAll 728 15792.3311 5.6130 25.0610 21.6928
MXNDArraySyncCopyToCPU 1576 12541.0166 0.0600 24.1980 7.9575
MXNDArrayGetDType 1576 0.5720 0.0000 0.0010 0.0004
MXNDArrayGetShape 4000 1.8770 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1696 14.8570 0.0070 0.0280 0.0088
MXNDArrayFree 18904 194.1100 0.0000 0.9140 0.0103
MXNet C API Calls 84496 571.8530 487.3580 571.8530 42.2475
MXNet C API Concurrency 168991 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1456 0.6870 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7280 3.7890 0.0000 0.0230 0.0005
MXNDArraySetGradState 7280 3.7180 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1456 0.6190 0.0000 0.0010 0.0004
MXAutogradBackwardEx 728 403.0790 0.5250 0.9450 0.5537
MXNDArrayGetContext 11752 5.1170 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24488 627.0420 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2912 7124.8359 0.7010 3.7500 2.4467
FullyConnected 5088 490.2090 0.0080 0.3430 0.0963
argmax 1696 33.5710 0.0050 0.0480 0.0198
_backward_pick 1456 23.1050 0.0130 0.0320 0.0159
_backward_mul_scalar 1456 6.3750 0.0030 0.0220 0.0044
DeleteVariable 62560 447.9280 0.0010 0.9010 0.0072
Flatten 1696 71.2360 0.0080 0.1120 0.0420
Activation 6784 315.0720 0.0030 1.9440 0.0464
SetValueOp 1456 9.4350 0.0050 0.0180 0.0065
pick 1456 11.4830 0.0060 0.0280 0.0079
Pooling 3392 4741.8569 0.0640 3.1720 1.3980
Convolution 3392 19327.8594 0.2260 13.2050 5.6981
CopyCPU2CPU 1696 9.8500 0.0030 0.0360 0.0058
log_softmax 1456 45.3280 0.0150 0.0550 0.0311
_backward_copy 1456 75.1940 0.0220 0.0760 0.0516
_backward_Convolution 2912 26939.5762 2.6240 12.5610 9.2512
_mul_scalar 1456 5.2120 0.0020 0.0230 0.0036
_backward_log_softmax 1456 24.8400 0.0090 0.0400 0.0171
mean 4608 88.1810 0.0080 0.0730 0.0191
_backward_mean 1456 16.0210 0.0080 0.1030 0.0110
_backward_FullyConnected 4368 836.4330 0.0280 0.5940 0.1915
WaitForVar 3152 28.0010 0.0060 0.0280 0.0089
broadcast_equal 1696 8.5140 0.0030 0.0110 0.0050
sgd_update 14560 144.4630 0.0020 0.0790 0.0099
_backward_Activation 5824 430.3730 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57546 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5094 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 728 2920.9551 0.3060 73.5870 4.0123
MXNDArrayCreateEx 849 29.2480 0.0310 0.0640 0.0344
MXNDArrayWaitAll 729 15814.1846 5.6130 25.0610 21.6930
MXNDArraySyncCopyToCPU 1578 12555.3438 0.0600 24.1980 7.9565
MXNDArrayGetDType 1578 0.5730 0.0000 0.0010 0.0004
MXNDArrayGetShape 4005 1.8810 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1698 14.8780 0.0070 0.0280 0.0088
MXNDArrayFree 18927 194.3540 0.0000 0.9140 0.0103
MXNet C API Calls 84603 571.9600 487.3580 571.9600 42.3010
MXNet C API Concurrency 169205 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1458 0.6870 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7290 3.7960 0.0000 0.0230 0.0005
MXNDArraySetGradState 7290 3.7230 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1458 0.6200 0.0000 0.0010 0.0004
MXAutogradBackwardEx 729 403.6310 0.5250 0.9450 0.5537
MXNDArrayGetContext 11766 5.1230 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24519 627.8650 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2916 7134.6401 0.7010 3.7500 2.4467
FullyConnected 5094 490.8060 0.0080 0.3430 0.0963
argmax 1698 33.6080 0.0050 0.0480 0.0198
_backward_pick 1458 23.1370 0.0130 0.0320 0.0159
_backward_mul_scalar 1458 6.3830 0.0030 0.0220 0.0044
DeleteVariable 62640 448.4870 0.0010 0.9010 0.0072
Flatten 1698 71.3140 0.0080 0.1120 0.0420
Activation 6792 315.4480 0.0030 1.9440 0.0464
SetValueOp 1458 9.4480 0.0050 0.0180 0.0065
pick 1458 11.4980 0.0060 0.0280 0.0079
Pooling 3396 4747.2979 0.0640 3.1720 1.3979
Convolution 3396 19350.0430 0.2260 13.2050 5.6979
CopyCPU2CPU 1698 9.8600 0.0030 0.0360 0.0058
log_softmax 1458 45.3880 0.0150 0.0550 0.0311
_backward_copy 1458 75.3010 0.0220 0.0760 0.0516
_backward_Convolution 2916 26976.5449 2.6240 12.5610 9.2512
_mul_scalar 1458 5.2180 0.0020 0.0230 0.0036
_backward_log_softmax 1458 24.8720 0.0090 0.0400 0.0171
mean 4614 88.3260 0.0080 0.0730 0.0191
_backward_mean 1458 16.0430 0.0080 0.1030 0.0110
_backward_FullyConnected 4374 837.5060 0.0280 0.5940 0.1915
WaitForVar 3156 28.0380 0.0060 0.0280 0.0089
broadcast_equal 1698 8.5250 0.0030 0.0110 0.0050
sgd_update 14580 144.6670 0.0020 0.0790 0.0099
_backward_Activation 5832 430.9670 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57620 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5100 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 729 2924.8730 0.3060 73.5870 4.0122
MXNDArrayCreateEx 850 29.2840 0.0310 0.0640 0.0345
MXNDArrayWaitAll 730 15836.0146 5.6130 25.0610 21.6932
MXNDArraySyncCopyToCPU 1580 12569.6689 0.0600 24.1980 7.9555
MXNDArrayGetDType 1580 0.5730 0.0000 0.0010 0.0004
MXNDArrayGetShape 4010 1.8830 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1700 14.8960 0.0070 0.0280 0.0088
MXNDArrayFree 18950 194.5910 0.0000 0.9140 0.0103
MXNet C API Calls 84710 572.0670 487.3580 572.0670 42.3545
MXNet C API Concurrency 169419 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1460 0.6880 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7300 3.8010 0.0000 0.0230 0.0005
MXNDArraySetGradState 7300 3.7300 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1460 0.6220 0.0000 0.0010 0.0004
MXAutogradBackwardEx 730 404.1790 0.5250 0.9450 0.5537
MXNDArrayGetContext 11780 5.1270 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24550 628.6840 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2920 7144.3320 0.7010 3.7500 2.4467
FullyConnected 5100 491.3280 0.0080 0.3430 0.0963
argmax 1700 33.6470 0.0050 0.0480 0.0198
_backward_pick 1460 23.1670 0.0130 0.0320 0.0159
_backward_mul_scalar 1460 6.3920 0.0030 0.0220 0.0044
DeleteVariable 62720 449.0200 0.0010 0.9010 0.0072
Flatten 1700 71.3920 0.0080 0.1120 0.0420
Activation 6800 315.8460 0.0030 1.9440 0.0464
SetValueOp 1460 9.4610 0.0050 0.0180 0.0065
pick 1460 11.5130 0.0060 0.0280 0.0079
Pooling 3400 4752.6440 0.0640 3.1720 1.3978
Convolution 3400 19372.2988 0.2260 13.2050 5.6977
CopyCPU2CPU 1700 9.8700 0.0030 0.0360 0.0058
log_softmax 1460 45.4480 0.0150 0.0550 0.0311
_backward_copy 1460 75.4080 0.0220 0.0760 0.0516
_backward_Convolution 2920 27013.5605 2.6240 12.5610 9.2512
_mul_scalar 1460 5.2250 0.0020 0.0230 0.0036
_backward_log_softmax 1460 24.9030 0.0090 0.0400 0.0171
mean 4620 88.4430 0.0080 0.0730 0.0191
_backward_mean 1460 16.0650 0.0080 0.1030 0.0110
_backward_FullyConnected 4380 838.5800 0.0280 0.5940 0.1915
WaitForVar 3160 28.0740 0.0060 0.0280 0.0089
broadcast_equal 1700 8.5340 0.0030 0.0110 0.0050
sgd_update 14600 144.8730 0.0020 0.0790 0.0099
_backward_Activation 5840 431.5670 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57694 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5106 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 730 2928.8101 0.3060 73.5870 4.0121
MXNDArrayCreateEx 851 29.3200 0.0310 0.0640 0.0345
MXNDArrayWaitAll 731 15857.7715 5.6130 25.0610 21.6933
MXNDArraySyncCopyToCPU 1582 12583.9023 0.0600 24.1980 7.9544
MXNDArrayGetDType 1582 0.5730 0.0000 0.0010 0.0004
MXNDArrayGetShape 4015 1.8850 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1702 14.9150 0.0070 0.0280 0.0088
MXNDArrayFree 18973 194.8310 0.0000 0.9140 0.0103
MXNet C API Calls 84817 572.1740 487.3580 572.1740 42.4080
MXNet C API Concurrency 169633 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1462 0.6890 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7310 3.8040 0.0000 0.0230 0.0005
MXNDArraySetGradState 7310 3.7350 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1462 0.6240 0.0000 0.0010 0.0004
MXAutogradBackwardEx 731 404.7580 0.5250 0.9450 0.5537
MXNDArrayGetContext 11794 5.1360 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24581 629.5060 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2924 7154.0601 0.7010 3.7500 2.4467
FullyConnected 5106 491.8740 0.0080 0.3430 0.0963
argmax 1702 33.6850 0.0050 0.0480 0.0198
_backward_pick 1462 23.1990 0.0130 0.0320 0.0159
_backward_mul_scalar 1462 6.4000 0.0030 0.0220 0.0044
DeleteVariable 62800 449.5570 0.0010 0.9010 0.0072
Flatten 1702 71.4700 0.0080 0.1120 0.0420
Activation 6808 316.2160 0.0030 1.9440 0.0464
SetValueOp 1462 9.4730 0.0050 0.0180 0.0065
pick 1462 11.5280 0.0060 0.0280 0.0079
Pooling 3404 4758.0332 0.0640 3.1720 1.3978
Convolution 3404 19394.5039 0.2260 13.2050 5.6976
CopyCPU2CPU 1702 9.8800 0.0030 0.0360 0.0058
log_softmax 1462 45.5070 0.0150 0.0550 0.0311
_backward_copy 1462 75.5140 0.0220 0.0760 0.0517
_backward_Convolution 2924 27050.3848 2.6240 12.5610 9.2512
_mul_scalar 1462 5.2320 0.0020 0.0230 0.0036
_backward_log_softmax 1462 24.9350 0.0090 0.0400 0.0171
mean 4626 88.5600 0.0080 0.0730 0.0191
_backward_mean 1462 16.0870 0.0080 0.1030 0.0110
_backward_FullyConnected 4386 839.6500 0.0280 0.5940 0.1914
WaitForVar 3164 28.1090 0.0060 0.0280 0.0089
broadcast_equal 1702 8.5440 0.0030 0.0110 0.0050
sgd_update 14620 145.0800 0.0020 0.0790 0.0099
_backward_Activation 5848 432.1320 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57768 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5112 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 731 2932.7371 0.3060 73.5870 4.0120
MXNDArrayCreateEx 852 29.3560 0.0310 0.0640 0.0345
MXNDArrayWaitAll 732 15879.4961 5.6130 25.0610 21.6933
MXNDArraySyncCopyToCPU 1584 12598.5156 0.0600 24.1980 7.9536
MXNDArrayGetDType 1584 0.5740 0.0000 0.0010 0.0004
MXNDArrayGetShape 4020 1.8870 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1704 14.9330 0.0070 0.0280 0.0088
MXNDArrayFree 18996 195.0700 0.0000 0.9140 0.0103
MXNet C API Calls 84924 572.2810 487.3580 572.2810 42.4615
MXNet C API Concurrency 169847 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1464 0.6890 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7320 3.8080 0.0000 0.0230 0.0005
MXNDArraySetGradState 7320 3.7400 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1464 0.6260 0.0000 0.0010 0.0004
MXAutogradBackwardEx 732 405.3020 0.5250 0.9450 0.5537
MXNDArrayGetContext 11808 5.1410 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24612 630.3550 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2928 7163.7578 0.7010 3.7500 2.4466
FullyConnected 5112 492.6340 0.0080 0.3430 0.0964
argmax 1704 33.7300 0.0050 0.0480 0.0198
_backward_pick 1464 23.2320 0.0130 0.0320 0.0159
_backward_mul_scalar 1464 6.4090 0.0030 0.0220 0.0044
DeleteVariable 62880 450.1180 0.0010 0.9010 0.0072
Flatten 1704 71.5510 0.0080 0.1120 0.0420
Activation 6816 316.6000 0.0030 1.9440 0.0464
SetValueOp 1464 9.4860 0.0050 0.0180 0.0065
pick 1464 11.5440 0.0060 0.0280 0.0079
Pooling 3408 4763.3828 0.0640 3.1720 1.3977
Convolution 3408 19416.7266 0.2260 13.2050 5.6974
CopyCPU2CPU 1704 9.8900 0.0030 0.0360 0.0058
log_softmax 1464 45.5780 0.0150 0.0550 0.0311
_backward_copy 1464 75.6160 0.0220 0.0760 0.0517
_backward_Convolution 2928 27087.2168 2.6240 12.5610 9.2511
_mul_scalar 1464 5.2390 0.0020 0.0230 0.0036
_backward_log_softmax 1464 24.9780 0.0090 0.0400 0.0171
mean 4632 88.6660 0.0080 0.0730 0.0191
_backward_mean 1464 16.1100 0.0080 0.1030 0.0110
_backward_FullyConnected 4392 841.0610 0.0280 0.5940 0.1915
WaitForVar 3168 28.1450 0.0060 0.0280 0.0089
broadcast_equal 1704 8.5530 0.0030 0.0110 0.0050
sgd_update 14640 145.3530 0.0020 0.0790 0.0099
_backward_Activation 5856 432.7430 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57842 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5118 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 732 2936.7029 0.3060 73.5870 4.0119
MXNDArrayCreateEx 853 29.3920 0.0310 0.0640 0.0345
MXNDArrayWaitAll 733 15901.3311 5.6130 25.0610 21.6935
MXNDArraySyncCopyToCPU 1586 12612.7773 0.0600 24.1980 7.9526
MXNDArrayGetDType 1586 0.5740 0.0000 0.0010 0.0004
MXNDArrayGetShape 4025 1.8910 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1706 14.9520 0.0070 0.0280 0.0088
MXNDArrayFree 19019 195.3100 0.0000 0.9140 0.0103
MXNet C API Calls 85031 572.3880 487.3580 572.3880 42.5150
MXNet C API Concurrency 170061 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1466 0.6900 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7330 3.8140 0.0000 0.0230 0.0005
MXNDArraySetGradState 7330 3.7450 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1466 0.6260 0.0000 0.0010 0.0004
MXAutogradBackwardEx 733 405.8530 0.5250 0.9450 0.5537
MXNDArrayGetContext 11822 5.1500 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24643 631.1780 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2932 7173.4380 0.7010 3.7500 2.4466
FullyConnected 5118 493.2450 0.0080 0.3430 0.0964
argmax 1706 33.7680 0.0050 0.0480 0.0198
_backward_pick 1466 23.2640 0.0130 0.0320 0.0159
_backward_mul_scalar 1466 6.4180 0.0030 0.0220 0.0044
DeleteVariable 62960 450.6590 0.0010 0.9010 0.0072
Flatten 1706 71.6280 0.0080 0.1120 0.0420
Activation 6824 316.9720 0.0030 1.9440 0.0464
SetValueOp 1466 9.4980 0.0050 0.0180 0.0065
pick 1466 11.5590 0.0060 0.0280 0.0079
Pooling 3412 4768.7300 0.0640 3.1720 1.3976
Convolution 3412 19438.9297 0.2260 13.2050 5.6972
CopyCPU2CPU 1706 9.9000 0.0030 0.0360 0.0058
log_softmax 1466 45.6380 0.0150 0.0550 0.0311
_backward_copy 1466 75.7220 0.0220 0.0760 0.0517
_backward_Convolution 2932 27124.2617 2.6240 12.5610 9.2511
_mul_scalar 1466 5.2460 0.0020 0.0230 0.0036
_backward_log_softmax 1466 25.0090 0.0090 0.0400 0.0171
mean 4638 88.7800 0.0080 0.0730 0.0191
_backward_mean 1466 16.1310 0.0080 0.1030 0.0110
_backward_FullyConnected 4398 842.1340 0.0280 0.5940 0.1915
WaitForVar 3172 28.1810 0.0060 0.0280 0.0089
broadcast_equal 1706 8.5630 0.0030 0.0110 0.0050
sgd_update 14660 145.5610 0.0020 0.0790 0.0099
_backward_Activation 5864 433.3390 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57916 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5124 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 733 2940.6350 0.3060 73.5870 4.0118
MXNDArrayCreateEx 854 29.4280 0.0310 0.0640 0.0345
MXNDArrayWaitAll 734 15923.2324 5.6130 25.0610 21.6938
MXNDArraySyncCopyToCPU 1588 12627.0859 0.0600 24.1980 7.9516
MXNDArrayGetDType 1588 0.5750 0.0000 0.0010 0.0004
MXNDArrayGetShape 4030 1.8940 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1708 14.9700 0.0070 0.0280 0.0088
MXNDArrayFree 19042 195.5530 0.0000 0.9140 0.0103
MXNet C API Calls 85138 572.4950 487.3580 572.4950 42.5685
MXNet C API Concurrency 170275 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1468 0.6910 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7340 3.8210 0.0000 0.0230 0.0005
MXNDArraySetGradState 7340 3.7490 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1468 0.6270 0.0000 0.0010 0.0004
MXAutogradBackwardEx 734 406.3980 0.5250 0.9450 0.5537
MXNDArrayGetContext 11836 5.1570 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24674 632.0260 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2936 7183.3130 0.7010 3.7500 2.4466
FullyConnected 5124 493.7790 0.0080 0.3430 0.0964
argmax 1708 33.8070 0.0050 0.0480 0.0198
_backward_pick 1468 23.2960 0.0130 0.0320 0.0159
_backward_mul_scalar 1468 6.4270 0.0030 0.0220 0.0044
DeleteVariable 63040 451.2100 0.0010 0.9010 0.0072
Flatten 1708 71.7080 0.0080 0.1120 0.0420
Activation 6832 317.3450 0.0030 1.9440 0.0464
SetValueOp 1468 9.5110 0.0050 0.0180 0.0065
pick 1468 11.5740 0.0060 0.0280 0.0079
Pooling 3416 4774.1299 0.0640 3.1720 1.3976
Convolution 3416 19461.1797 0.2260 13.2050 5.6971
CopyCPU2CPU 1708 9.9110 0.0030 0.0360 0.0058
log_softmax 1468 45.6970 0.0150 0.0550 0.0311
_backward_copy 1468 75.8280 0.0220 0.0760 0.0517
_backward_Convolution 2936 27161.3340 2.6240 12.5610 9.2511
_mul_scalar 1468 5.2520 0.0020 0.0230 0.0036
_backward_log_softmax 1468 25.0410 0.0090 0.0400 0.0171
mean 4644 88.8940 0.0080 0.0730 0.0191
_backward_mean 1468 16.1520 0.0080 0.1030 0.0110
_backward_FullyConnected 4404 843.2080 0.0280 0.5940 0.1915
WaitForVar 3176 28.2170 0.0060 0.0280 0.0089
broadcast_equal 1708 8.5720 0.0030 0.0110 0.0050
sgd_update 14680 145.7110 0.0020 0.0790 0.0099
_backward_Activation 5872 433.9670 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 57990 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5130 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 734 2944.5569 0.3060 73.5870 4.0117
MXNDArrayCreateEx 855 29.4640 0.0310 0.0640 0.0345
MXNDArrayWaitAll 735 15944.9453 5.6130 25.0610 21.6938
MXNDArraySyncCopyToCPU 1590 12641.3633 0.0600 24.1980 7.9505
MXNDArrayGetDType 1590 0.5760 0.0000 0.0010 0.0004
MXNDArrayGetShape 4035 1.8960 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1710 14.9880 0.0070 0.0280 0.0088
MXNDArrayFree 19065 195.7870 0.0000 0.9140 0.0103
MXNet C API Calls 85245 572.6020 487.3580 572.6020 42.6220
MXNet C API Concurrency 170489 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1470 0.6910 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7350 3.8250 0.0000 0.0230 0.0005
MXNDArraySetGradState 7350 3.7520 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1470 0.6280 0.0000 0.0010 0.0004
MXAutogradBackwardEx 735 406.9450 0.5250 0.9450 0.5537
MXNDArrayGetContext 11850 5.1610 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24705 632.8520 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2940 7192.9468 0.7010 3.7500 2.4466
FullyConnected 5130 494.3260 0.0080 0.3430 0.0964
argmax 1710 33.8460 0.0050 0.0480 0.0198
_backward_pick 1470 23.3270 0.0130 0.0320 0.0159
_backward_mul_scalar 1470 6.4360 0.0030 0.0220 0.0044
DeleteVariable 63120 451.7470 0.0010 0.9010 0.0072
Flatten 1710 71.7830 0.0080 0.1120 0.0420
Activation 6840 317.7140 0.0030 1.9440 0.0464
SetValueOp 1470 9.5240 0.0050 0.0180 0.0065
pick 1470 11.5900 0.0060 0.0280 0.0079
Pooling 3420 4779.4858 0.0640 3.1720 1.3975
Convolution 3420 19483.4609 0.2260 13.2050 5.6969
CopyCPU2CPU 1710 9.9210 0.0030 0.0360 0.0058
log_softmax 1470 45.7560 0.0150 0.0550 0.0311
_backward_copy 1470 75.9340 0.0220 0.0760 0.0517
_backward_Convolution 2940 27198.1445 2.6240 12.5610 9.2511
_mul_scalar 1470 5.2590 0.0020 0.0230 0.0036
_backward_log_softmax 1470 25.0730 0.0090 0.0400 0.0171
mean 4650 89.0070 0.0080 0.0730 0.0191
_backward_mean 1470 16.1740 0.0080 0.1030 0.0110
_backward_FullyConnected 4410 844.2810 0.0280 0.5940 0.1914
WaitForVar 3180 28.2540 0.0060 0.0280 0.0089
broadcast_equal 1710 8.5810 0.0030 0.0110 0.0050
sgd_update 14700 145.9130 0.0020 0.0790 0.0099
_backward_Activation 5880 434.5470 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58064 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5136 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 735 2948.4829 0.3060 73.5870 4.0115
MXNDArrayCreateEx 856 29.5000 0.0310 0.0640 0.0345
MXNDArrayWaitAll 736 15966.6426 5.6130 25.0610 21.6938
MXNDArraySyncCopyToCPU 1592 12655.9463 0.0600 24.1980 7.9497
MXNDArrayGetDType 1592 0.5770 0.0000 0.0010 0.0004
MXNDArrayGetShape 4040 1.8990 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1712 15.0060 0.0070 0.0280 0.0088
MXNDArrayFree 19088 196.0260 0.0000 0.9140 0.0103
MXNet C API Calls 85352 572.7090 487.3580 572.7090 42.6755
MXNet C API Concurrency 170703 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1472 0.6920 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7360 3.8270 0.0000 0.0230 0.0005
MXNDArraySetGradState 7360 3.7580 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1472 0.6300 0.0000 0.0010 0.0004
MXAutogradBackwardEx 736 407.4920 0.5250 0.9450 0.5537
MXNDArrayGetContext 11864 5.1700 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24736 633.7020 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2944 7202.7129 0.7010 3.7500 2.4466
FullyConnected 5136 495.0760 0.0080 0.3430 0.0964
argmax 1712 33.8910 0.0050 0.0480 0.0198
_backward_pick 1472 23.3590 0.0130 0.0320 0.0159
_backward_mul_scalar 1472 6.4450 0.0030 0.0220 0.0044
DeleteVariable 63200 452.2850 0.0010 0.9010 0.0072
Flatten 1712 71.8620 0.0080 0.1120 0.0420
Activation 6848 318.1290 0.0030 1.9440 0.0465
SetValueOp 1472 9.5370 0.0050 0.0180 0.0065
pick 1472 11.6070 0.0060 0.0280 0.0079
Pooling 3424 4784.8330 0.0640 3.1720 1.3974
Convolution 3424 19505.6309 0.2260 13.2050 5.6967
CopyCPU2CPU 1712 9.9300 0.0030 0.0360 0.0058
log_softmax 1472 45.8290 0.0150 0.0550 0.0311
_backward_copy 1472 76.0370 0.0220 0.0760 0.0517
_backward_Convolution 2944 27234.8477 2.6240 12.5610 9.2510
_mul_scalar 1472 5.2660 0.0020 0.0230 0.0036
_backward_log_softmax 1472 25.1140 0.0090 0.0400 0.0171
mean 4656 89.1120 0.0080 0.0730 0.0191
_backward_mean 1472 16.1960 0.0080 0.1030 0.0110
_backward_FullyConnected 4416 845.6960 0.0280 0.5940 0.1915
WaitForVar 3184 28.2900 0.0060 0.0280 0.0089
broadcast_equal 1712 8.5900 0.0030 0.0110 0.0050
sgd_update 14720 146.1860 0.0020 0.0790 0.0099
_backward_Activation 5888 435.1520 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58138 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5142 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 736 2952.4131 0.3060 73.5870 4.0114
MXNDArrayCreateEx 857 29.5350 0.0310 0.0640 0.0345
MXNDArrayWaitAll 737 15988.4307 5.6130 25.0610 21.6939
MXNDArraySyncCopyToCPU 1594 12670.2275 0.0600 24.1980 7.9487
MXNDArrayGetDType 1594 0.5790 0.0000 0.0010 0.0004
MXNDArrayGetShape 4045 1.8990 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1714 15.0250 0.0070 0.0280 0.0088
MXNDArrayFree 19111 196.2680 0.0000 0.9140 0.0103
MXNet C API Calls 85459 572.8160 487.3580 572.8160 42.7290
MXNet C API Concurrency 170917 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1474 0.6930 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7370 3.8300 0.0000 0.0230 0.0005
MXNDArraySetGradState 7370 3.7600 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1474 0.6310 0.0000 0.0010 0.0004
MXAutogradBackwardEx 737 408.0460 0.5250 0.9450 0.5537
MXNDArrayGetContext 11878 5.1750 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24767 634.5260 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2948 7212.4160 0.7010 3.7500 2.4465
FullyConnected 5142 495.6310 0.0080 0.3430 0.0964
argmax 1714 33.9300 0.0050 0.0480 0.0198
_backward_pick 1474 23.3890 0.0130 0.0320 0.0159
_backward_mul_scalar 1474 6.4540 0.0030 0.0220 0.0044
DeleteVariable 63280 452.8500 0.0010 0.9010 0.0072
Flatten 1714 71.9430 0.0080 0.1120 0.0420
Activation 6856 318.5000 0.0030 1.9440 0.0465
SetValueOp 1474 9.5490 0.0050 0.0180 0.0065
pick 1474 11.6210 0.0060 0.0280 0.0079
Pooling 3428 4790.1831 0.0640 3.1720 1.3974
Convolution 3428 19527.7969 0.2260 13.2050 5.6966
CopyCPU2CPU 1714 9.9410 0.0030 0.0360 0.0058
log_softmax 1474 45.8880 0.0150 0.0550 0.0311
_backward_copy 1474 76.1850 0.0220 0.0760 0.0517
_backward_Convolution 2948 27271.8086 2.6240 12.5610 9.2510
_mul_scalar 1474 5.2730 0.0020 0.0230 0.0036
_backward_log_softmax 1474 25.1460 0.0090 0.0400 0.0171
mean 4662 89.2520 0.0080 0.0730 0.0191
_backward_mean 1474 16.2180 0.0080 0.1030 0.0110
_backward_FullyConnected 4422 846.7730 0.0280 0.5940 0.1915
WaitForVar 3188 28.3250 0.0060 0.0280 0.0089
broadcast_equal 1714 8.5990 0.0030 0.0110 0.0050
sgd_update 14740 146.3880 0.0020 0.0790 0.0099
_backward_Activation 5896 435.7290 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58212 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5148 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 737 2956.3140 0.3060 73.5870 4.0113
MXNDArrayCreateEx 858 29.5720 0.0310 0.0640 0.0345
MXNDArrayWaitAll 738 16010.2939 5.6130 25.0610 21.6942
MXNDArraySyncCopyToCPU 1596 12684.5293 0.0600 24.1980 7.9477
MXNDArrayGetDType 1596 0.5810 0.0000 0.0010 0.0004
MXNDArrayGetShape 4050 1.9020 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1716 15.0420 0.0070 0.0280 0.0088
MXNDArrayFree 19134 196.5030 0.0000 0.9140 0.0103
MXNet C API Calls 85566 572.9230 487.3580 572.9230 42.7825
MXNet C API Concurrency 171131 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1476 0.6940 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7380 3.8360 0.0000 0.0230 0.0005
MXNDArraySetGradState 7380 3.7660 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1476 0.6320 0.0000 0.0010 0.0004
MXAutogradBackwardEx 738 408.6170 0.5250 0.9450 0.5537
MXNDArrayGetContext 11892 5.1790 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24798 635.3520 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2952 7222.1768 0.7010 3.7500 2.4465
FullyConnected 5148 496.1800 0.0080 0.3430 0.0964
argmax 1716 33.9670 0.0050 0.0480 0.0198
_backward_pick 1476 23.4190 0.0130 0.0320 0.0159
_backward_mul_scalar 1476 6.4630 0.0030 0.0220 0.0044
DeleteVariable 63360 453.4070 0.0010 0.9010 0.0072
Flatten 1716 72.0220 0.0080 0.1120 0.0420
Activation 6864 318.8710 0.0030 1.9440 0.0465
SetValueOp 1476 9.5620 0.0050 0.0180 0.0065
pick 1476 11.6360 0.0060 0.0280 0.0079
Pooling 3432 4795.5981 0.0640 3.1720 1.3973
Convolution 3432 19550.0293 0.2260 13.2050 5.6964
CopyCPU2CPU 1716 9.9510 0.0030 0.0360 0.0058
log_softmax 1476 45.9470 0.0150 0.0550 0.0311
_backward_copy 1476 76.2890 0.0220 0.0760 0.0517
_backward_Convolution 2952 27308.8535 2.6240 12.5610 9.2510
_mul_scalar 1476 5.2800 0.0020 0.0230 0.0036
_backward_log_softmax 1476 25.1780 0.0090 0.0400 0.0171
mean 4668 89.3710 0.0080 0.0730 0.0191
_backward_mean 1476 16.2400 0.0080 0.1030 0.0110
_backward_FullyConnected 4428 847.8460 0.0280 0.5940 0.1915
WaitForVar 3192 28.3600 0.0060 0.0280 0.0089
broadcast_equal 1716 8.6090 0.0030 0.0110 0.0050
sgd_update 14760 146.5910 0.0020 0.0790 0.0099
_backward_Activation 5904 436.3380 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58286 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5154 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 738 2960.2380 0.3060 73.5870 4.0112
MXNDArrayCreateEx 859 29.6080 0.0310 0.0640 0.0345
MXNDArrayWaitAll 739 16031.9746 5.6130 25.0610 21.6941
MXNDArraySyncCopyToCPU 1598 12698.8154 0.0600 24.1980 7.9467
MXNDArrayGetDType 1598 0.5810 0.0000 0.0010 0.0004
MXNDArrayGetShape 4055 1.9040 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1718 15.0600 0.0070 0.0280 0.0088
MXNDArrayFree 19157 196.7630 0.0000 0.9140 0.0103
MXNet C API Calls 85673 573.0300 487.3580 573.0300 42.8360
MXNet C API Concurrency 171345 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1478 0.6960 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7390 3.8390 0.0000 0.0230 0.0005
MXNDArraySetGradState 7390 3.7710 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1478 0.6330 0.0000 0.0010 0.0004
MXAutogradBackwardEx 739 409.1650 0.5250 0.9450 0.5537
MXNDArrayGetContext 11906 5.1870 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24829 636.1720 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2956 7231.8672 0.7010 3.7500 2.4465
FullyConnected 5154 496.7140 0.0080 0.3430 0.0964
argmax 1718 34.0050 0.0050 0.0480 0.0198
_backward_pick 1478 23.4510 0.0130 0.0320 0.0159
_backward_mul_scalar 1478 6.4720 0.0030 0.0220 0.0044
DeleteVariable 63440 453.9970 0.0010 0.9010 0.0072
Flatten 1718 72.1010 0.0080 0.1120 0.0420
Activation 6872 319.2870 0.0030 1.9440 0.0465
SetValueOp 1478 9.5750 0.0050 0.0180 0.0065
pick 1478 11.6520 0.0060 0.0280 0.0079
Pooling 3436 4800.9458 0.0640 3.1720 1.3972
Convolution 3436 19572.2188 0.2260 13.2050 5.6962
CopyCPU2CPU 1718 9.9610 0.0030 0.0360 0.0058
log_softmax 1478 46.0060 0.0150 0.0550 0.0311
_backward_copy 1478 76.3950 0.0220 0.0760 0.0517
_backward_Convolution 2956 27345.6074 2.6240 12.5610 9.2509
_mul_scalar 1478 5.2870 0.0020 0.0230 0.0036
_backward_log_softmax 1478 25.2110 0.0090 0.0400 0.0171
mean 4674 89.4900 0.0080 0.0730 0.0191
_backward_mean 1478 16.2620 0.0080 0.1030 0.0110
_backward_FullyConnected 4434 848.9200 0.0280 0.5940 0.1915
WaitForVar 3196 28.3950 0.0060 0.0280 0.0089
broadcast_equal 1718 8.6190 0.0030 0.0110 0.0050
sgd_update 14780 146.7420 0.0020 0.0790 0.0099
_backward_Activation 5912 436.9990 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58360 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5160 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 739 2964.1621 0.3060 73.5870 4.0110
MXNDArrayCreateEx 860 29.6440 0.0310 0.0640 0.0345
MXNDArrayWaitAll 740 16053.6484 5.6130 25.0610 21.6941
MXNDArraySyncCopyToCPU 1600 12713.3828 0.0600 24.1980 7.9459
MXNDArrayGetDType 1600 0.5810 0.0000 0.0010 0.0004
MXNDArrayGetShape 4060 1.9060 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1720 15.0780 0.0070 0.0280 0.0088
MXNDArrayFree 19180 197.0000 0.0000 0.9140 0.0103
MXNet C API Calls 85780 573.1370 487.3580 573.1370 42.8895
MXNet C API Concurrency 171559 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1480 0.6960 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7400 3.8430 0.0000 0.0230 0.0005
MXNDArraySetGradState 7400 3.7760 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1480 0.6340 0.0000 0.0010 0.0004
MXAutogradBackwardEx 740 409.7190 0.5250 0.9450 0.5537
MXNDArrayGetContext 11920 5.1950 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24860 636.9940 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2960 7241.4902 0.7010 3.7500 2.4464
FullyConnected 5160 497.4730 0.0080 0.3430 0.0964
argmax 1720 34.0510 0.0050 0.0480 0.0198
_backward_pick 1480 23.4830 0.0130 0.0320 0.0159
_backward_mul_scalar 1480 6.4810 0.0030 0.0220 0.0044
DeleteVariable 63520 454.5330 0.0010 0.9010 0.0072
Flatten 1720 72.1800 0.0080 0.1120 0.0420
Activation 6880 319.6680 0.0030 1.9440 0.0465
SetValueOp 1480 9.5880 0.0050 0.0180 0.0065
pick 1480 11.6680 0.0060 0.0280 0.0079
Pooling 3440 4806.3188 0.0640 3.1720 1.3972
Convolution 3440 19594.4395 0.2260 13.2050 5.6961
CopyCPU2CPU 1720 9.9710 0.0030 0.0360 0.0058
log_softmax 1480 46.0780 0.0150 0.0550 0.0311
_backward_copy 1480 76.4980 0.0220 0.0760 0.0517
_backward_Convolution 2960 27382.3457 2.6240 12.5610 9.2508
_mul_scalar 1480 5.2940 0.0020 0.0230 0.0036
_backward_log_softmax 1480 25.2530 0.0090 0.0400 0.0171
mean 4680 89.5960 0.0080 0.0730 0.0191
_backward_mean 1480 16.2840 0.0080 0.1030 0.0110
_backward_FullyConnected 4440 850.3270 0.0280 0.5940 0.1915
WaitForVar 3200 28.4320 0.0060 0.0280 0.0089
broadcast_equal 1720 8.6290 0.0030 0.0110 0.0050
sgd_update 14800 147.0150 0.0020 0.0790 0.0099
_backward_Activation 5920 437.6070 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58434 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5166 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 740 2968.0891 0.3060 73.5870 4.0109
MXNDArrayCreateEx 861 29.6800 0.0310 0.0640 0.0345
MXNDArrayWaitAll 741 16075.4453 5.6130 25.0610 21.6943
MXNDArraySyncCopyToCPU 1602 12727.6670 0.0600 24.1980 7.9449
MXNDArrayGetDType 1602 0.5810 0.0000 0.0010 0.0004
MXNDArrayGetShape 4065 1.9090 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1722 15.0960 0.0070 0.0280 0.0088
MXNDArrayFree 19203 197.2350 0.0000 0.9140 0.0103
MXNet C API Calls 85887 573.2440 487.3580 573.2440 42.9430
MXNet C API Concurrency 171773 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1482 0.6970 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7410 3.8490 0.0000 0.0230 0.0005
MXNDArraySetGradState 7410 3.7810 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1482 0.6350 0.0000 0.0010 0.0004
MXAutogradBackwardEx 741 410.2640 0.5250 0.9450 0.5537
MXNDArrayGetContext 11934 5.2000 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24891 637.8510 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2964 7251.2710 0.7010 3.7500 2.4464
FullyConnected 5166 497.9980 0.0080 0.3430 0.0964
argmax 1722 34.0890 0.0050 0.0480 0.0198
_backward_pick 1482 23.5150 0.0130 0.0320 0.0159
_backward_mul_scalar 1482 6.4900 0.0030 0.0220 0.0044
DeleteVariable 63600 455.0760 0.0010 0.9010 0.0072
Flatten 1722 72.2590 0.0080 0.1120 0.0420
Activation 6888 320.0380 0.0030 1.9440 0.0465
SetValueOp 1482 9.6020 0.0050 0.0180 0.0065
pick 1482 11.6840 0.0060 0.0280 0.0079
Pooling 3444 4811.7061 0.0640 3.1720 1.3971
Convolution 3444 19616.6250 0.2260 13.2050 5.6959
CopyCPU2CPU 1722 9.9800 0.0030 0.0360 0.0058
log_softmax 1482 46.1380 0.0150 0.0550 0.0311
_backward_copy 1482 76.6030 0.0220 0.0760 0.0517
_backward_Convolution 2964 27419.3027 2.6240 12.5610 9.2508
_mul_scalar 1482 5.3010 0.0020 0.0230 0.0036
_backward_log_softmax 1482 25.2850 0.0090 0.0400 0.0171
mean 4686 89.7120 0.0080 0.0730 0.0191
_backward_mean 1482 16.3060 0.0080 0.1030 0.0110
_backward_FullyConnected 4446 851.3980 0.0280 0.5940 0.1915
WaitForVar 3204 28.4680 0.0060 0.0280 0.0089
broadcast_equal 1722 8.6390 0.0030 0.0110 0.0050
sgd_update 14820 147.1670 0.0020 0.0790 0.0099
_backward_Activation 5928 438.2510 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58508 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5172 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 741 2972.0071 0.3060 73.5870 4.0108
MXNDArrayCreateEx 862 29.7160 0.0310 0.0640 0.0345
MXNDArrayWaitAll 742 16097.3057 5.6130 25.0610 21.6945
MXNDArraySyncCopyToCPU 1604 12741.9766 0.0600 24.1980 7.9439
MXNDArrayGetDType 1604 0.5810 0.0000 0.0010 0.0004
MXNDArrayGetShape 4070 1.9110 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1724 15.1130 0.0070 0.0280 0.0088
MXNDArrayFree 19226 197.4700 0.0000 0.9140 0.0103
MXNet C API Calls 85994 573.3510 487.3580 573.3510 42.9965
MXNet C API Concurrency 171987 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1484 0.6980 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7420 3.8540 0.0000 0.0230 0.0005
MXNDArraySetGradState 7420 3.7880 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1484 0.6360 0.0000 0.0010 0.0004
MXAutogradBackwardEx 742 410.8110 0.5250 0.9450 0.5537
MXNDArrayGetContext 11948 5.2050 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24922 638.6980 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2968 7261.0591 0.7010 3.7500 2.4464
FullyConnected 5172 498.5270 0.0080 0.3430 0.0964
argmax 1724 34.1270 0.0050 0.0480 0.0198
_backward_pick 1484 23.5460 0.0130 0.0320 0.0159
_backward_mul_scalar 1484 6.4990 0.0030 0.0220 0.0044
DeleteVariable 63680 455.6080 0.0010 0.9010 0.0072
Flatten 1724 72.3380 0.0080 0.1120 0.0420
Activation 6896 320.4100 0.0030 1.9440 0.0465
SetValueOp 1484 9.6150 0.0050 0.0180 0.0065
pick 1484 11.6980 0.0060 0.0280 0.0079
Pooling 3448 4817.0640 0.0640 3.1720 1.3971
Convolution 3448 19638.8926 0.2260 13.2050 5.6957
CopyCPU2CPU 1724 9.9910 0.0030 0.0360 0.0058
log_softmax 1484 46.1970 0.0150 0.0550 0.0311
_backward_copy 1484 76.7100 0.0220 0.0760 0.0517
_backward_Convolution 2968 27456.3262 2.6240 12.5610 9.2508
_mul_scalar 1484 5.3080 0.0020 0.0230 0.0036
_backward_log_softmax 1484 25.3170 0.0090 0.0400 0.0171
mean 4692 89.8290 0.0080 0.0730 0.0191
_backward_mean 1484 16.3280 0.0080 0.1030 0.0110
_backward_FullyConnected 4452 852.4720 0.0280 0.5940 0.1915
WaitForVar 3208 28.5040 0.0060 0.0280 0.0089
broadcast_equal 1724 8.6480 0.0030 0.0110 0.0050
sgd_update 14840 147.3160 0.0020 0.0790 0.0099
_backward_Activation 5936 438.8800 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58582 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5178 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 742 2975.9280 0.3060 73.5870 4.0107
MXNDArrayCreateEx 863 29.7510 0.0310 0.0640 0.0345
MXNDArrayWaitAll 743 16118.9639 5.6130 25.0610 21.6944
MXNDArraySyncCopyToCPU 1606 12756.2480 0.0600 24.1980 7.9429
MXNDArrayGetDType 1606 0.5830 0.0000 0.0010 0.0004
MXNDArrayGetShape 4075 1.9110 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1726 15.1320 0.0070 0.0280 0.0088
MXNDArrayFree 19249 197.7050 0.0000 0.9140 0.0103
MXNet C API Calls 86101 573.4580 487.3580 573.4580 43.0500
MXNet C API Concurrency 172201 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1486 0.7000 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7430 3.8610 0.0000 0.0230 0.0005
MXNDArraySetGradState 7430 3.7940 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1486 0.6380 0.0000 0.0010 0.0004
MXAutogradBackwardEx 743 411.3560 0.5250 0.9450 0.5536
MXNDArrayGetContext 11962 5.2130 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24953 639.5240 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2972 7270.7158 0.7010 3.7500 2.4464
FullyConnected 5178 499.0530 0.0080 0.3430 0.0964
argmax 1726 34.1640 0.0050 0.0480 0.0198
_backward_pick 1486 23.5780 0.0130 0.0320 0.0159
_backward_mul_scalar 1486 6.5070 0.0030 0.0220 0.0044
DeleteVariable 63760 456.1540 0.0010 0.9010 0.0072
Flatten 1726 72.4160 0.0080 0.1120 0.0420
Activation 6904 320.7800 0.0030 1.9440 0.0465
SetValueOp 1486 9.6290 0.0050 0.0180 0.0065
pick 1486 11.7140 0.0060 0.0280 0.0079
Pooling 3452 4822.4038 0.0640 3.1720 1.3970
Convolution 3452 19661.1445 0.2260 13.2050 5.6956
CopyCPU2CPU 1726 10.0020 0.0030 0.0360 0.0058
log_softmax 1486 46.2570 0.0150 0.0550 0.0311
_backward_copy 1486 76.8170 0.0220 0.0760 0.0517
_backward_Convolution 2972 27493.0977 2.6240 12.5610 9.2507
_mul_scalar 1486 5.3150 0.0020 0.0230 0.0036
_backward_log_softmax 1486 25.3490 0.0090 0.0400 0.0171
mean 4698 89.9440 0.0080 0.0730 0.0191
_backward_mean 1486 16.3500 0.0080 0.1030 0.0110
_backward_FullyConnected 4458 853.5470 0.0280 0.5940 0.1915
WaitForVar 3212 28.5400 0.0060 0.0280 0.0089
broadcast_equal 1726 8.6570 0.0030 0.0110 0.0050
sgd_update 14860 147.5170 0.0020 0.0790 0.0099
_backward_Activation 5944 439.4620 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58656 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5184 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 743 2979.8540 0.3060 73.5870 4.0106
MXNDArrayCreateEx 864 29.7880 0.0310 0.0640 0.0345
MXNDArrayWaitAll 744 16140.6465 5.6130 25.0610 21.6944
MXNDArraySyncCopyToCPU 1608 12770.5400 0.0600 24.1980 7.9419
MXNDArrayGetDType 1608 0.5840 0.0000 0.0010 0.0004
MXNDArrayGetShape 4080 1.9120 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1728 15.1500 0.0070 0.0280 0.0088
MXNDArrayFree 19272 197.9450 0.0000 0.9140 0.0103
MXNet C API Calls 86208 573.5650 487.3580 573.5650 43.1035
MXNet C API Concurrency 172415 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1488 0.7010 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7440 3.8670 0.0000 0.0230 0.0005
MXNDArraySetGradState 7440 3.8000 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1488 0.6380 0.0000 0.0010 0.0004
MXAutogradBackwardEx 744 411.9280 0.5250 0.9450 0.5537
MXNDArrayGetContext 11976 5.2160 0.0000 0.0070 0.0004
MXImperativeInvokeEx 24984 640.3630 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2976 7280.4141 0.7010 3.7500 2.4464
FullyConnected 5184 499.8110 0.0080 0.3430 0.0964
argmax 1728 34.2100 0.0050 0.0480 0.0198
_backward_pick 1488 23.6100 0.0130 0.0320 0.0159
_backward_mul_scalar 1488 6.5160 0.0030 0.0220 0.0044
DeleteVariable 63840 456.7140 0.0010 0.9010 0.0072
Flatten 1728 72.4950 0.0080 0.1120 0.0420
Activation 6912 321.1580 0.0030 1.9440 0.0465
SetValueOp 1488 9.6420 0.0050 0.0180 0.0065
pick 1488 11.7300 0.0060 0.0280 0.0079
Pooling 3456 4827.7441 0.0640 3.1720 1.3969
Convolution 3456 19683.3574 0.2260 13.2050 5.6954
CopyCPU2CPU 1728 10.0120 0.0030 0.0360 0.0058
log_softmax 1488 46.3310 0.0150 0.0550 0.0311
_backward_copy 1488 76.9190 0.0220 0.0760 0.0517
_backward_Convolution 2976 27529.8340 2.6240 12.5610 9.2506
_mul_scalar 1488 5.3220 0.0020 0.0230 0.0036
_backward_log_softmax 1488 25.3900 0.0090 0.0400 0.0171
mean 4704 90.0510 0.0080 0.0730 0.0191
_backward_mean 1488 16.3730 0.0080 0.1030 0.0110
_backward_FullyConnected 4464 855.0190 0.0280 0.5940 0.1915
WaitForVar 3216 28.5750 0.0060 0.0280 0.0089
broadcast_equal 1728 8.6670 0.0030 0.0110 0.0050
sgd_update 14880 147.7830 0.0020 0.0790 0.0099
_backward_Activation 5952 440.0480 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58730 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5190 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 744 2983.7810 0.3060 73.5870 4.0105
MXNDArrayCreateEx 865 29.8240 0.0310 0.0640 0.0345
MXNDArrayWaitAll 745 16162.4238 5.6130 25.0610 21.6945
MXNDArraySyncCopyToCPU 1610 12784.6719 0.0600 24.1980 7.9408
MXNDArrayGetDType 1610 0.5840 0.0000 0.0010 0.0004
MXNDArrayGetShape 4085 1.9170 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1730 15.1690 0.0070 0.0280 0.0088
MXNDArrayFree 19295 198.1840 0.0000 0.9140 0.0103
MXNet C API Calls 86315 573.6720 487.3580 573.6720 43.1570
MXNet C API Concurrency 172629 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1490 0.7010 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7450 3.8730 0.0000 0.0230 0.0005
MXNDArraySetGradState 7450 3.8040 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1490 0.6390 0.0000 0.0010 0.0004
MXAutogradBackwardEx 745 412.4790 0.5250 0.9450 0.5537
MXNDArrayGetContext 11990 5.2250 0.0000 0.0070 0.0004
MXImperativeInvokeEx 25015 641.2010 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2980 7290.1538 0.7010 3.7500 2.4464
FullyConnected 5190 500.3630 0.0080 0.3430 0.0964
argmax 1730 34.2480 0.0050 0.0480 0.0198
_backward_pick 1490 23.6410 0.0130 0.0320 0.0159
_backward_mul_scalar 1490 6.5250 0.0030 0.0220 0.0044
DeleteVariable 63920 457.2460 0.0010 0.9010 0.0072
Flatten 1730 72.5730 0.0080 0.1120 0.0419
Activation 6920 321.5440 0.0030 1.9440 0.0465
SetValueOp 1490 9.6550 0.0050 0.0180 0.0065
pick 1490 11.7460 0.0060 0.0280 0.0079
Pooling 3460 4833.1260 0.0640 3.1720 1.3969
Convolution 3460 19705.4902 0.2260 13.2050 5.6952
CopyCPU2CPU 1730 10.0220 0.0030 0.0360 0.0058
log_softmax 1490 46.3900 0.0150 0.0550 0.0311
_backward_copy 1490 77.0230 0.0220 0.0760 0.0517
_backward_Convolution 2980 27566.8223 2.6240 12.5610 9.2506
_mul_scalar 1490 5.3290 0.0020 0.0230 0.0036
_backward_log_softmax 1490 25.4220 0.0090 0.0400 0.0171
mean 4710 90.1670 0.0080 0.0730 0.0191
_backward_mean 1490 16.3940 0.0080 0.1030 0.0110
_backward_FullyConnected 4470 856.0890 0.0280 0.5940 0.1915
WaitForVar 3220 28.6120 0.0060 0.0280 0.0089
broadcast_equal 1730 8.6770 0.0030 0.0110 0.0050
sgd_update 14900 147.9850 0.0020 0.0790 0.0099
_backward_Activation 5960 440.6190 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58804 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5196 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 745 2987.7061 0.3060 73.5870 4.0103
MXNDArrayCreateEx 866 29.8610 0.0310 0.0640 0.0345
MXNDArrayWaitAll 746 16184.2383 5.6130 25.0610 21.6947
MXNDArraySyncCopyToCPU 1612 12798.9971 0.0600 24.1980 7.9398
MXNDArrayGetDType 1612 0.5840 0.0000 0.0010 0.0004
MXNDArrayGetShape 4090 1.9190 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1732 15.1880 0.0070 0.0280 0.0088
MXNDArrayFree 19318 198.4200 0.0000 0.9140 0.0103
MXNet C API Calls 86422 573.7790 487.3580 573.7790 43.2105
MXNet C API Concurrency 172843 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1492 0.7020 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7460 3.8780 0.0000 0.0230 0.0005
MXNDArraySetGradState 7460 3.8070 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1492 0.6400 0.0000 0.0010 0.0004
MXAutogradBackwardEx 746 413.0290 0.5250 0.9450 0.5537
MXNDArrayGetContext 12004 5.2340 0.0000 0.0070 0.0004
MXImperativeInvokeEx 25046 642.0140 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2984 7299.8481 0.7010 3.7500 2.4463
FullyConnected 5196 500.8900 0.0080 0.3430 0.0964
argmax 1732 34.2860 0.0050 0.0480 0.0198
_backward_pick 1492 23.6740 0.0130 0.0320 0.0159
_backward_mul_scalar 1492 6.5320 0.0030 0.0220 0.0044
DeleteVariable 64000 457.8050 0.0010 0.9010 0.0072
Flatten 1732 72.6500 0.0080 0.1120 0.0419
Activation 6928 321.9130 0.0030 1.9440 0.0465
SetValueOp 1492 9.6690 0.0050 0.0180 0.0065
pick 1492 11.7620 0.0060 0.0280 0.0079
Pooling 3464 4838.4668 0.0640 3.1720 1.3968
Convolution 3464 19727.7383 0.2260 13.2050 5.6951
CopyCPU2CPU 1732 10.0330 0.0030 0.0360 0.0058
log_softmax 1492 46.4500 0.0150 0.0550 0.0311
_backward_copy 1492 77.1320 0.0220 0.0760 0.0517
_backward_Convolution 2984 27603.8906 2.6240 12.5610 9.2506
_mul_scalar 1492 5.3350 0.0020 0.0230 0.0036
_backward_log_softmax 1492 25.4550 0.0090 0.0400 0.0171
mean 4716 90.2840 0.0080 0.0730 0.0191
_backward_mean 1492 16.4150 0.0080 0.1030 0.0110
_backward_FullyConnected 4476 857.1560 0.0280 0.5940 0.1915
WaitForVar 3224 28.6490 0.0060 0.0280 0.0089
broadcast_equal 1732 8.6870 0.0030 0.0110 0.0050
sgd_update 14920 148.1960 0.0020 0.0790 0.0099
_backward_Activation 5968 441.2020 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58878 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5202 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 746 2991.6521 0.3060 73.5870 4.0103
MXNDArrayCreateEx 867 29.8980 0.0310 0.0640 0.0345
MXNDArrayWaitAll 747 16205.9609 5.6130 25.0610 21.6947
MXNDArraySyncCopyToCPU 1614 12813.3232 0.0600 24.1980 7.9389
MXNDArrayGetDType 1614 0.5850 0.0000 0.0010 0.0004
MXNDArrayGetShape 4095 1.9210 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1734 15.2070 0.0070 0.0280 0.0088
MXNDArrayFree 19341 198.6650 0.0000 0.9140 0.0103
MXNet C API Calls 86529 573.8860 487.3580 573.8860 43.2640
MXNet C API Concurrency 173057 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1494 0.7030 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7470 3.8850 0.0000 0.0230 0.0005
MXNDArraySetGradState 7470 3.8120 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1494 0.6410 0.0000 0.0010 0.0004
MXAutogradBackwardEx 747 413.6110 0.5250 0.9450 0.5537
MXNDArrayGetContext 12018 5.2400 0.0000 0.0070 0.0004
MXImperativeInvokeEx 25077 642.8520 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2988 7309.5742 0.7010 3.7500 2.4463
FullyConnected 5202 501.4560 0.0080 0.3430 0.0964
argmax 1734 34.3250 0.0050 0.0480 0.0198
_backward_pick 1494 23.7070 0.0130 0.0320 0.0159
_backward_mul_scalar 1494 6.5400 0.0030 0.0220 0.0044
DeleteVariable 64080 458.3500 0.0010 0.9010 0.0072
Flatten 1734 72.7430 0.0080 0.1120 0.0420
Activation 6936 322.2940 0.0030 1.9440 0.0465
SetValueOp 1494 9.6830 0.0050 0.0180 0.0065
pick 1494 11.7770 0.0060 0.0280 0.0079
Pooling 3468 4843.8691 0.0640 3.1720 1.3967
Convolution 3468 19750.0605 0.2260 13.2050 5.6949
CopyCPU2CPU 1734 10.0440 0.0030 0.0360 0.0058
log_softmax 1494 46.5100 0.0150 0.0550 0.0311
_backward_copy 1494 77.2380 0.0220 0.0760 0.0517
_backward_Convolution 2988 27640.6680 2.6240 12.5610 9.2506
_mul_scalar 1494 5.3430 0.0020 0.0230 0.0036
_backward_log_softmax 1494 25.4870 0.0090 0.0400 0.0171
mean 4722 90.4030 0.0080 0.0730 0.0191
_backward_mean 1494 16.4370 0.0080 0.1030 0.0110
_backward_FullyConnected 4482 858.2430 0.0280 0.5940 0.1915
WaitForVar 3228 28.6840 0.0060 0.0280 0.0089
broadcast_equal 1734 8.6970 0.0030 0.0110 0.0050
sgd_update 14940 148.3990 0.0020 0.0790 0.0099
_backward_Activation 5976 441.7890 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 58952 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5208 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 747 2995.6289 0.3060 73.5870 4.0102
MXNDArrayCreateEx 868 29.9360 0.0310 0.0640 0.0345
MXNDArrayWaitAll 748 16227.6758 5.6130 25.0610 21.6948
MXNDArraySyncCopyToCPU 1616 12828.0117 0.0600 24.1980 7.9381
MXNDArrayGetDType 1616 0.5860 0.0000 0.0010 0.0004
MXNDArrayGetShape 4100 1.9240 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1736 15.2250 0.0070 0.0280 0.0088
MXNDArrayFree 19364 198.9160 0.0000 0.9140 0.0103
MXNet C API Calls 86636 573.9930 487.3580 573.9930 43.3175
MXNet C API Concurrency 173271 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1496 0.7030 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7480 3.8910 0.0000 0.0230 0.0005
MXNDArraySetGradState 7480 3.8160 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1496 0.6430 0.0000 0.0010 0.0004
MXAutogradBackwardEx 748 414.1780 0.5250 0.9450 0.5537
MXNDArrayGetContext 12032 5.2470 0.0000 0.0070 0.0004
MXImperativeInvokeEx 25108 643.7170 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2992 7319.3198 0.7010 3.7500 2.4463
FullyConnected 5208 502.2190 0.0080 0.3430 0.0964
argmax 1736 34.3700 0.0050 0.0480 0.0198
_backward_pick 1496 23.7440 0.0130 0.0320 0.0159
_backward_mul_scalar 1496 6.5480 0.0030 0.0220 0.0044
DeleteVariable 64160 458.9330 0.0010 0.9010 0.0072
Flatten 1736 72.8420 0.0080 0.1120 0.0420
Activation 6944 322.6930 0.0030 1.9440 0.0465
SetValueOp 1496 9.6980 0.0050 0.0180 0.0065
pick 1496 11.7950 0.0060 0.0280 0.0079
Pooling 3472 4849.2651 0.0640 3.1720 1.3967
Convolution 3472 19772.3281 0.2260 13.2050 5.6948
CopyCPU2CPU 1736 10.0550 0.0030 0.0360 0.0058
log_softmax 1496 46.5830 0.0150 0.0550 0.0311
_backward_copy 1496 77.3410 0.0220 0.0760 0.0517
_backward_Convolution 2992 27677.4121 2.6240 12.5610 9.2505
_mul_scalar 1496 5.3500 0.0020 0.0230 0.0036
_backward_log_softmax 1496 25.5300 0.0090 0.0400 0.0171
mean 4728 90.5150 0.0080 0.0730 0.0191
_backward_mean 1496 16.4600 0.0080 0.1030 0.0110
_backward_FullyConnected 4488 859.6870 0.0280 0.5940 0.1916
WaitForVar 3232 28.7200 0.0060 0.0280 0.0089
broadcast_equal 1736 8.7080 0.0030 0.0110 0.0050
sgd_update 14960 148.6750 0.0020 0.0790 0.0099
_backward_Activation 5984 442.4210 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59026 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5214 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 748 2999.6760 0.3060 73.5870 4.0103
MXNDArrayCreateEx 869 29.9740 0.0310 0.0640 0.0345
MXNDArrayWaitAll 749 16249.4570 5.6130 25.0610 21.6949
MXNDArraySyncCopyToCPU 1618 12842.2852 0.0600 24.1980 7.9371
MXNDArrayGetDType 1618 0.5870 0.0000 0.0010 0.0004
MXNDArrayGetShape 4105 1.9270 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1738 15.2460 0.0070 0.0280 0.0088
MXNDArrayFree 19387 199.2050 0.0000 0.9140 0.0103
MXNet C API Calls 86743 574.1000 487.3580 574.1000 43.3710
MXNet C API Concurrency 173485 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1498 0.7040 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7490 3.8970 0.0000 0.0230 0.0005
MXNDArraySetGradState 7490 3.8250 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1498 0.6430 0.0000 0.0010 0.0004
MXAutogradBackwardEx 749 414.7850 0.5250 0.9450 0.5538
MXNDArrayGetContext 12046 5.2520 0.0000 0.0070 0.0004
MXImperativeInvokeEx 25139 644.5930 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 2996 7328.9800 0.7010 3.7500 2.4463
FullyConnected 5214 502.7840 0.0080 0.3430 0.0964
argmax 1738 34.4100 0.0050 0.0480 0.0198
_backward_pick 1498 23.7800 0.0130 0.0320 0.0159
_backward_mul_scalar 1498 6.5580 0.0030 0.0220 0.0044
DeleteVariable 64240 459.5480 0.0010 0.9010 0.0072
Flatten 1738 72.9650 0.0080 0.1120 0.0420
Activation 6952 323.0880 0.0030 1.9440 0.0465
SetValueOp 1498 9.7130 0.0050 0.0180 0.0065
pick 1498 11.8110 0.0060 0.0280 0.0079
Pooling 3476 4854.6641 0.0640 3.1720 1.3966
Convolution 3476 19794.6406 0.2260 13.2050 5.6947
CopyCPU2CPU 1738 10.0670 0.0030 0.0360 0.0058
log_softmax 1498 46.6460 0.0150 0.0550 0.0311
_backward_copy 1498 77.4470 0.0220 0.0760 0.0517
_backward_Convolution 2996 27714.4453 2.6240 12.5610 9.2505
_mul_scalar 1498 5.3570 0.0020 0.0230 0.0036
_backward_log_softmax 1498 25.5630 0.0090 0.0400 0.0171
mean 4734 90.6620 0.0080 0.0730 0.0192
_backward_mean 1498 16.4830 0.0080 0.1030 0.0110
_backward_FullyConnected 4494 860.7920 0.0280 0.5940 0.1915
WaitForVar 3236 28.7560 0.0060 0.0280 0.0089
broadcast_equal 1738 8.7180 0.0030 0.0110 0.0050
sgd_update 14980 148.8820 0.0020 0.0790 0.0099
_backward_Activation 5992 443.0110 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59100 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5220 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 749 3003.6609 0.3060 73.5870 4.0102
MXNDArrayCreateEx 870 30.0180 0.0310 0.0640 0.0345
MXNDArrayWaitAll 750 16271.4297 5.6130 25.0610 21.6952
MXNDArraySyncCopyToCPU 1620 12856.6797 0.0600 24.1980 7.9362
MXNDArrayGetDType 1620 0.5870 0.0000 0.0010 0.0004
MXNDArrayGetShape 4110 1.9300 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1740 15.2640 0.0070 0.0280 0.0088
MXNDArrayFree 19410 199.4590 0.0000 0.9140 0.0103
MXNet C API Calls 86850 574.2070 487.3580 574.2070 43.4245
MXNet C API Concurrency 173699 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1500 0.7050 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7500 3.9010 0.0000 0.0230 0.0005
MXNDArraySetGradState 7500 3.8280 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1500 0.6450 0.0000 0.0010 0.0004
MXAutogradBackwardEx 750 415.3640 0.5250 0.9450 0.5538
MXNDArrayGetContext 12060 5.2570 0.0000 0.0070 0.0004
MXImperativeInvokeEx 25170 645.4350 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3000 7338.8271 0.7010 3.7500 2.4463
FullyConnected 5220 503.3810 0.0080 0.3430 0.0964
argmax 1740 34.4500 0.0050 0.0480 0.0198
_backward_pick 1500 23.8170 0.0130 0.0320 0.0159
_backward_mul_scalar 1500 6.5680 0.0030 0.0220 0.0044
DeleteVariable 64320 460.1400 0.0010 0.9010 0.0072
Flatten 1740 73.0810 0.0080 0.1120 0.0420
Activation 6960 323.4740 0.0030 1.9440 0.0465
SetValueOp 1500 9.7280 0.0050 0.0180 0.0065
pick 1500 11.8270 0.0060 0.0280 0.0079
Pooling 3480 4860.0361 0.0640 3.1720 1.3966
Convolution 3480 19817.0098 0.2260 13.2050 5.6945
CopyCPU2CPU 1740 10.0800 0.0030 0.0360 0.0058
log_softmax 1500 46.7080 0.0150 0.0550 0.0311
_backward_copy 1500 77.5550 0.0220 0.0760 0.0517
_backward_Convolution 3000 27751.5547 2.6240 12.5610 9.2505
_mul_scalar 1500 5.3650 0.0020 0.0230 0.0036
_backward_log_softmax 1500 25.5960 0.0090 0.0400 0.0171
mean 4740 90.7800 0.0080 0.0730 0.0192
_backward_mean 1500 16.5060 0.0080 0.1030 0.0110
_backward_FullyConnected 4500 861.8970 0.0280 0.5940 0.1915
WaitForVar 3240 28.7930 0.0060 0.0280 0.0089
broadcast_equal 1740 8.7290 0.0030 0.0110 0.0050
sgd_update 15000 149.0960 0.0020 0.0790 0.0099
_backward_Activation 6000 443.6280 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59174 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5226 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 750 3004.0249 0.3060 73.5870 4.0054
MXNDArrayCreateEx 871 30.0560 0.0310 0.0640 0.0345
MXNDArrayWaitAll 751 16293.1367 5.6130 25.0610 21.6953
MXNDArraySyncCopyToCPU 1622 12870.7422 0.0600 24.1980 7.9351
MXNDArrayGetDType 1622 0.5870 0.0000 0.0010 0.0004
MXNDArrayGetShape 4115 1.9330 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1742 15.2840 0.0070 0.0280 0.0088
MXNDArrayFree 19433 199.7320 0.0000 0.9140 0.0103
MXNet C API Calls 86957 574.3140 487.3580 574.3140 43.4780
MXNet C API Concurrency 173913 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1502 0.7050 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7510 3.9060 0.0000 0.0230 0.0005
MXNDArraySetGradState 7510 3.8340 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1502 0.6460 0.0000 0.0010 0.0004
MXAutogradBackwardEx 751 415.9720 0.5250 0.9450 0.5539
MXNDArrayGetContext 12074 5.2830 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25201 646.3400 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3004 7348.5459 0.7010 3.7500 2.4463
FullyConnected 5226 503.9260 0.0080 0.3430 0.0964
argmax 1742 34.4900 0.0050 0.0480 0.0198
_backward_pick 1502 23.8510 0.0130 0.0320 0.0159
_backward_mul_scalar 1502 6.5770 0.0030 0.0220 0.0044
DeleteVariable 64400 460.7390 0.0010 0.9010 0.0072
Flatten 1742 73.1860 0.0080 0.1120 0.0420
Activation 6968 323.8600 0.0030 1.9440 0.0465
SetValueOp 1502 9.7430 0.0050 0.0180 0.0065
pick 1502 11.8430 0.0060 0.0280 0.0079
Pooling 3484 4865.4629 0.0640 3.1720 1.3965
Convolution 3484 19839.3105 0.2260 13.2050 5.6944
CopyCPU2CPU 1742 10.0910 0.0030 0.0360 0.0058
log_softmax 1502 46.7690 0.0150 0.0550 0.0311
_backward_copy 1502 77.6600 0.0220 0.0760 0.0517
_backward_Convolution 3004 27788.3086 2.6240 12.5610 9.2504
_mul_scalar 1502 5.3720 0.0020 0.0230 0.0036
_backward_log_softmax 1502 25.6310 0.0090 0.0400 0.0171
mean 4746 90.8950 0.0080 0.0730 0.0192
_backward_mean 1502 16.5300 0.0080 0.1030 0.0110
_backward_FullyConnected 4506 862.9790 0.0280 0.5940 0.1915
WaitForVar 3244 28.8280 0.0060 0.0280 0.0089
broadcast_equal 1742 8.7400 0.0030 0.0110 0.0050
sgd_update 15020 149.3220 0.0020 0.0790 0.0099
_backward_Activation 6008 444.2160 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59248 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5232 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 751 3007.9580 0.3060 73.5870 4.0053
MXNDArrayCreateEx 872 30.1150 0.0310 0.0640 0.0345
MXNDArrayWaitAll 752 16314.8525 5.6130 25.0610 21.6953
MXNDArraySyncCopyToCPU 1624 12885.2803 0.0600 24.1980 7.9343
MXNDArrayGetDType 1624 0.5880 0.0000 0.0010 0.0004
MXNDArrayGetShape 4120 1.9350 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1744 15.3020 0.0070 0.0280 0.0088
MXNDArrayFree 19456 199.9750 0.0000 0.9140 0.0103
MXNet C API Calls 87064 574.4210 487.3580 574.4210 43.5315
MXNet C API Concurrency 174127 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1504 0.7070 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7520 3.9130 0.0000 0.0230 0.0005
MXNDArraySetGradState 7520 3.8390 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1504 0.6460 0.0000 0.0010 0.0004
MXAutogradBackwardEx 752 416.5330 0.5250 0.9450 0.5539
MXNDArrayGetContext 12088 5.2900 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25232 647.1650 0.0080 0.1450 0.0256
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3008 7358.2119 0.7010 3.7500 2.4462
FullyConnected 5232 504.6590 0.0080 0.3430 0.0965
argmax 1744 34.5340 0.0050 0.0480 0.0198
_backward_pick 1504 23.8850 0.0130 0.0320 0.0159
_backward_mul_scalar 1504 6.5860 0.0030 0.0220 0.0044
DeleteVariable 64480 461.3000 0.0010 0.9010 0.0072
Flatten 1744 73.2640 0.0080 0.1120 0.0420
Activation 6976 324.2380 0.0030 1.9440 0.0465
SetValueOp 1504 9.7570 0.0050 0.0180 0.0065
pick 1504 11.8610 0.0060 0.0280 0.0079
Pooling 3488 4870.7681 0.0640 3.1720 1.3964
Convolution 3488 19861.5566 0.2260 13.2050 5.6943
CopyCPU2CPU 1744 10.1020 0.0030 0.0360 0.0058
log_softmax 1504 46.8410 0.0150 0.0550 0.0311
_backward_copy 1504 77.7620 0.0220 0.0760 0.0517
_backward_Convolution 3008 27825.0664 2.6240 12.5610 9.2504
_mul_scalar 1504 5.3780 0.0020 0.0230 0.0036
_backward_log_softmax 1504 25.6730 0.0090 0.0400 0.0171
mean 4752 91.0050 0.0080 0.0730 0.0192
_backward_mean 1504 16.5510 0.0080 0.1030 0.0110
_backward_FullyConnected 4512 864.4510 0.0280 0.5940 0.1916
WaitForVar 3248 28.8640 0.0060 0.0280 0.0089
broadcast_equal 1744 8.7500 0.0030 0.0110 0.0050
sgd_update 15040 149.5970 0.0020 0.0790 0.0099
_backward_Activation 6016 444.8110 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59322 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5238 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 752 3011.8950 0.3060 73.5870 4.0052
MXNDArrayCreateEx 873 30.1510 0.0310 0.0640 0.0345
MXNDArrayWaitAll 753 16336.6787 5.6130 25.0610 21.6955
MXNDArraySyncCopyToCPU 1626 12899.5400 0.0600 24.1980 7.9333
MXNDArrayGetDType 1626 0.5890 0.0000 0.0010 0.0004
MXNDArrayGetShape 4125 1.9380 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1746 15.3210 0.0070 0.0280 0.0088
MXNDArrayFree 19479 200.2270 0.0000 0.9140 0.0103
MXNet C API Calls 87171 574.5280 487.3580 574.5280 43.5850
MXNet C API Concurrency 174341 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1506 0.7080 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7530 3.9150 0.0000 0.0230 0.0005
MXNDArraySetGradState 7530 3.8450 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1506 0.6470 0.0000 0.0010 0.0004
MXAutogradBackwardEx 753 417.0900 0.5250 0.9450 0.5539
MXNDArrayGetContext 12102 5.2960 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25263 648.0000 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3012 7367.9302 0.7010 3.7500 2.4462
FullyConnected 5238 505.1970 0.0080 0.3430 0.0964
argmax 1746 34.5720 0.0050 0.0480 0.0198
_backward_pick 1506 23.9380 0.0130 0.0320 0.0159
_backward_mul_scalar 1506 6.5950 0.0030 0.0220 0.0044
DeleteVariable 64560 461.8570 0.0010 0.9010 0.0072
Flatten 1746 73.3450 0.0080 0.1120 0.0420
Activation 6984 324.6260 0.0030 1.9440 0.0465
SetValueOp 1506 9.7700 0.0050 0.0180 0.0065
pick 1506 11.8770 0.0060 0.0280 0.0079
Pooling 3492 4876.1499 0.0640 3.1720 1.3964
Convolution 3492 19883.7344 0.2260 13.2050 5.6941
CopyCPU2CPU 1746 10.1130 0.0030 0.0360 0.0058
log_softmax 1506 46.9020 0.0150 0.0550 0.0311
_backward_copy 1506 77.8680 0.0220 0.0760 0.0517
_backward_Convolution 3012 27862.1035 2.6240 12.5610 9.2504
_mul_scalar 1506 5.3860 0.0020 0.0230 0.0036
_backward_log_softmax 1506 25.7050 0.0090 0.0400 0.0171
mean 4758 91.1220 0.0080 0.0730 0.0192
_backward_mean 1506 16.5720 0.0080 0.1030 0.0110
_backward_FullyConnected 4518 865.5260 0.0280 0.5940 0.1916
WaitForVar 3252 28.8990 0.0060 0.0280 0.0089
broadcast_equal 1746 8.7600 0.0030 0.0110 0.0050
sgd_update 15060 149.8030 0.0020 0.0790 0.0099
_backward_Activation 6024 445.3790 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59396 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5244 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 753 3015.8330 0.3060 73.5870 4.0051
MXNDArrayCreateEx 874 30.1880 0.0310 0.0640 0.0345
MXNDArrayWaitAll 754 16358.5781 5.6130 25.0610 21.6957
MXNDArraySyncCopyToCPU 1628 12913.8555 0.0600 24.1980 7.9323
MXNDArrayGetDType 1628 0.5890 0.0000 0.0010 0.0004
MXNDArrayGetShape 4130 1.9420 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1748 15.3400 0.0070 0.0280 0.0088
MXNDArrayFree 19502 200.4680 0.0000 0.9140 0.0103
MXNet C API Calls 87278 574.6350 487.3580 574.6350 43.6385
MXNet C API Concurrency 174555 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1508 0.7080 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7540 3.9190 0.0000 0.0230 0.0005
MXNDArraySetGradState 7540 3.8510 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1508 0.6480 0.0000 0.0010 0.0004
MXAutogradBackwardEx 754 417.6440 0.5250 0.9450 0.5539
MXNDArrayGetContext 12116 5.3030 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25294 648.8280 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3016 7377.7051 0.7010 3.7500 2.4462
FullyConnected 5244 505.7290 0.0080 0.3430 0.0964
argmax 1748 34.6100 0.0050 0.0480 0.0198
_backward_pick 1508 23.9690 0.0130 0.0320 0.0159
_backward_mul_scalar 1508 6.6040 0.0030 0.0220 0.0044
DeleteVariable 64640 462.4020 0.0010 0.9010 0.0072
Flatten 1748 73.4310 0.0080 0.1120 0.0420
Activation 6992 324.9940 0.0030 1.9440 0.0465
SetValueOp 1508 9.7830 0.0050 0.0180 0.0065
pick 1508 11.8920 0.0060 0.0280 0.0079
Pooling 3496 4881.5039 0.0640 3.1720 1.3963
Convolution 3496 19906.0371 0.2260 13.2050 5.6939
CopyCPU2CPU 1748 10.1230 0.0030 0.0360 0.0058
log_softmax 1508 46.9620 0.0150 0.0550 0.0311
_backward_copy 1508 77.9750 0.0220 0.0760 0.0517
_backward_Convolution 3016 27899.1973 2.6240 12.5610 9.2504
_mul_scalar 1508 5.3930 0.0020 0.0230 0.0036
_backward_log_softmax 1508 25.7370 0.0090 0.0400 0.0171
mean 4764 91.2420 0.0080 0.0730 0.0192
_backward_mean 1508 16.5940 0.0080 0.1030 0.0110
_backward_FullyConnected 4524 866.6480 0.0280 0.5940 0.1916
WaitForVar 3256 28.9350 0.0060 0.0280 0.0089
broadcast_equal 1748 8.7710 0.0030 0.0110 0.0050
sgd_update 15080 149.9570 0.0020 0.0790 0.0099
_backward_Activation 6032 446.0270 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59470 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5250 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 754 3019.7649 0.3060 73.5870 4.0050
MXNDArrayCreateEx 875 30.2270 0.0310 0.0640 0.0345
MXNDArrayWaitAll 755 16380.2988 5.6130 25.0610 21.6958
MXNDArraySyncCopyToCPU 1630 12928.0986 0.0600 24.1980 7.9313
MXNDArrayGetDType 1630 0.5890 0.0000 0.0010 0.0004
MXNDArrayGetShape 4135 1.9460 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1750 15.3580 0.0070 0.0280 0.0088
MXNDArrayFree 19525 200.7100 0.0000 0.9140 0.0103
MXNet C API Calls 87385 574.7420 487.3580 574.7420 43.6920
MXNet C API Concurrency 174769 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1510 0.7090 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7550 3.9240 0.0000 0.0230 0.0005
MXNDArraySetGradState 7550 3.8550 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1510 0.6490 0.0000 0.0010 0.0004
MXAutogradBackwardEx 755 418.1960 0.5250 0.9450 0.5539
MXNDArrayGetContext 12130 5.3090 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25325 649.6830 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3020 7387.3989 0.7010 3.7500 2.4462
FullyConnected 5250 506.2730 0.0080 0.3430 0.0964
argmax 1750 34.6490 0.0050 0.0480 0.0198
_backward_pick 1510 24.0000 0.0130 0.0320 0.0159
_backward_mul_scalar 1510 6.6130 0.0030 0.0220 0.0044
DeleteVariable 64720 462.9560 0.0010 0.9010 0.0072
Flatten 1750 73.5110 0.0080 0.1120 0.0420
Activation 7000 325.3630 0.0030 1.9440 0.0465
SetValueOp 1510 9.7970 0.0050 0.0180 0.0065
pick 1510 11.9070 0.0060 0.0280 0.0079
Pooling 3500 4886.9111 0.0640 3.1720 1.3963
Convolution 3500 19928.1855 0.2260 13.2050 5.6938
CopyCPU2CPU 1750 10.1330 0.0030 0.0360 0.0058
log_softmax 1510 47.0220 0.0150 0.0550 0.0311
_backward_copy 1510 78.0800 0.0220 0.0760 0.0517
_backward_Convolution 3020 27935.9785 2.6240 12.5610 9.2503
_mul_scalar 1510 5.4000 0.0020 0.0230 0.0036
_backward_log_softmax 1510 25.7690 0.0090 0.0400 0.0171
mean 4770 91.3800 0.0080 0.0730 0.0192
_backward_mean 1510 16.6160 0.0080 0.1030 0.0110
_backward_FullyConnected 4530 867.7230 0.0280 0.5940 0.1916
WaitForVar 3260 28.9700 0.0060 0.0280 0.0089
broadcast_equal 1750 8.7810 0.0030 0.0110 0.0050
sgd_update 15100 150.1650 0.0020 0.0790 0.0099
_backward_Activation 6040 446.6090 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59544 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5256 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 755 3023.6841 0.3060 73.5870 4.0049
MXNDArrayCreateEx 876 30.2630 0.0310 0.0640 0.0345
MXNDArrayWaitAll 756 16402.0488 5.6130 25.0610 21.6958
MXNDArraySyncCopyToCPU 1632 12942.6973 0.0600 24.1980 7.9306
MXNDArrayGetDType 1632 0.5890 0.0000 0.0010 0.0004
MXNDArrayGetShape 4140 1.9490 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1752 15.3770 0.0070 0.0280 0.0088
MXNDArrayFree 19548 200.9500 0.0000 0.9140 0.0103
MXNet C API Calls 87492 574.8490 487.3580 574.8490 43.7455
MXNet C API Concurrency 174983 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1512 0.7100 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7560 3.9270 0.0000 0.0230 0.0005
MXNDArraySetGradState 7560 3.8570 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1512 0.6510 0.0000 0.0010 0.0004
MXAutogradBackwardEx 756 418.7490 0.5250 0.9450 0.5539
MXNDArrayGetContext 12144 5.3160 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25356 650.5060 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3024 7397.1279 0.7010 3.7500 2.4461
FullyConnected 5256 507.0140 0.0080 0.3430 0.0965
argmax 1752 34.6940 0.0050 0.0480 0.0198
_backward_pick 1512 24.0330 0.0130 0.0320 0.0159
_backward_mul_scalar 1512 6.6210 0.0030 0.0220 0.0044
DeleteVariable 64800 463.5010 0.0010 0.9010 0.0072
Flatten 1752 73.5900 0.0080 0.1120 0.0420
Activation 7008 325.7750 0.0030 1.9440 0.0465
SetValueOp 1512 9.8090 0.0050 0.0180 0.0065
pick 1512 11.9230 0.0060 0.0280 0.0079
Pooling 3504 4892.2222 0.0640 3.1720 1.3962
Convolution 3504 19950.4180 0.2260 13.2050 5.6936
CopyCPU2CPU 1752 10.1430 0.0030 0.0360 0.0058
log_softmax 1512 47.0960 0.0150 0.0550 0.0311
_backward_copy 1512 78.1810 0.0220 0.0760 0.0517
_backward_Convolution 3024 27972.7480 2.6240 12.5610 9.2502
_mul_scalar 1512 5.4070 0.0020 0.0230 0.0036
_backward_log_softmax 1512 25.8100 0.0090 0.0400 0.0171
mean 4776 91.4890 0.0080 0.0730 0.0192
_backward_mean 1512 16.6380 0.0080 0.1030 0.0110
_backward_FullyConnected 4536 869.1710 0.0280 0.5940 0.1916
WaitForVar 3264 29.0050 0.0060 0.0280 0.0089
broadcast_equal 1752 8.7910 0.0030 0.0110 0.0050
sgd_update 15120 150.4400 0.0020 0.0790 0.0099
_backward_Activation 6048 447.2140 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59618 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5262 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 756 3027.5750 0.3060 73.5870 4.0047
MXNDArrayCreateEx 877 30.2990 0.0310 0.0640 0.0345
MXNDArrayWaitAll 757 16423.9395 5.6130 25.0610 21.6961
MXNDArraySyncCopyToCPU 1634 12956.9268 0.0600 24.1980 7.9296
MXNDArrayGetDType 1634 0.5910 0.0000 0.0010 0.0004
MXNDArrayGetShape 4145 1.9530 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1754 15.3960 0.0070 0.0280 0.0088
MXNDArrayFree 19571 201.1930 0.0000 0.9140 0.0103
MXNet C API Calls 87599 574.9560 487.3580 574.9560 43.7990
MXNet C API Concurrency 175197 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1514 0.7120 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7570 3.9310 0.0000 0.0230 0.0005
MXNDArraySetGradState 7570 3.8630 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1514 0.6510 0.0000 0.0010 0.0004
MXAutogradBackwardEx 757 419.3270 0.5250 0.9450 0.5539
MXNDArrayGetContext 12158 5.3210 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25387 651.3330 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3028 7406.8379 0.7010 3.7500 2.4461
FullyConnected 5262 507.5650 0.0080 0.3430 0.0965
argmax 1754 34.7330 0.0050 0.0480 0.0198
_backward_pick 1514 24.0650 0.0130 0.0320 0.0159
_backward_mul_scalar 1514 6.6300 0.0030 0.0220 0.0044
DeleteVariable 64880 464.0730 0.0010 0.9010 0.0072
Flatten 1754 73.6690 0.0080 0.1120 0.0420
Activation 7016 326.1460 0.0030 1.9440 0.0465
SetValueOp 1514 9.8220 0.0050 0.0180 0.0065
pick 1514 11.9380 0.0060 0.0280 0.0079
Pooling 3508 4897.5801 0.0640 3.1720 1.3961
Convolution 3508 19972.6289 0.2260 13.2050 5.6935
CopyCPU2CPU 1754 10.1520 0.0030 0.0360 0.0058
log_softmax 1514 47.1560 0.0150 0.0550 0.0311
_backward_copy 1514 78.2870 0.0220 0.0760 0.0517
_backward_Convolution 3028 28009.8789 2.6240 12.5610 9.2503
_mul_scalar 1514 5.4140 0.0020 0.0230 0.0036
_backward_log_softmax 1514 25.8430 0.0090 0.0400 0.0171
mean 4782 91.6080 0.0080 0.0730 0.0192
_backward_mean 1514 16.6590 0.0080 0.1030 0.0110
_backward_FullyConnected 4542 870.2440 0.0280 0.5940 0.1916
WaitForVar 3268 29.0410 0.0060 0.0280 0.0089
broadcast_equal 1754 8.8020 0.0030 0.0110 0.0050
sgd_update 15140 150.6450 0.0020 0.0790 0.0100
_backward_Activation 6056 447.7890 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59692 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5268 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 757 3031.5510 0.3060 73.5870 4.0047
MXNDArrayCreateEx 878 30.3360 0.0310 0.0640 0.0346
MXNDArrayWaitAll 758 16445.7500 5.6130 25.0610 21.6962
MXNDArraySyncCopyToCPU 1636 12971.1992 0.0600 24.1980 7.9286
MXNDArrayGetDType 1636 0.5920 0.0000 0.0010 0.0004
MXNDArrayGetShape 4150 1.9560 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1756 15.4150 0.0070 0.0280 0.0088
MXNDArrayFree 19594 201.4390 0.0000 0.9140 0.0103
MXNet C API Calls 87706 575.0630 487.3580 575.0630 43.8525
MXNet C API Concurrency 175411 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1516 0.7140 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7580 3.9370 0.0000 0.0230 0.0005
MXNDArraySetGradState 7580 3.8670 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1516 0.6530 0.0000 0.0010 0.0004
MXAutogradBackwardEx 758 419.8880 0.5250 0.9450 0.5539
MXNDArrayGetContext 12172 5.3270 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25418 652.1710 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3032 7416.4751 0.7010 3.7500 2.4461
FullyConnected 5268 508.1030 0.0080 0.3430 0.0965
argmax 1756 34.7720 0.0050 0.0480 0.0198
_backward_pick 1516 24.0970 0.0130 0.0320 0.0159
_backward_mul_scalar 1516 6.6400 0.0030 0.0220 0.0044
DeleteVariable 64960 464.6180 0.0010 0.9010 0.0072
Flatten 1756 73.7620 0.0080 0.1120 0.0420
Activation 7024 326.5260 0.0030 1.9440 0.0465
SetValueOp 1516 9.8350 0.0050 0.0180 0.0065
pick 1516 11.9540 0.0060 0.0280 0.0079
Pooling 3512 4902.9258 0.0640 3.1720 1.3960
Convolution 3512 19994.8984 0.2260 13.2050 5.6933
CopyCPU2CPU 1756 10.1630 0.0030 0.0360 0.0058
log_softmax 1516 47.2180 0.0150 0.0550 0.0311
_backward_copy 1516 78.3930 0.0220 0.0760 0.0517
_backward_Convolution 3032 28046.9336 2.6240 12.5610 9.2503
_mul_scalar 1516 5.4210 0.0020 0.0230 0.0036
_backward_log_softmax 1516 25.8760 0.0090 0.0400 0.0171
mean 4788 91.7270 0.0080 0.0730 0.0192
_backward_mean 1516 16.6810 0.0080 0.1030 0.0110
_backward_FullyConnected 4548 871.3230 0.0280 0.5940 0.1916
WaitForVar 3272 29.0770 0.0060 0.0280 0.0089
broadcast_equal 1756 8.8130 0.0030 0.0110 0.0050
sgd_update 15160 150.8510 0.0020 0.0790 0.0100
_backward_Activation 6064 448.3640 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59766 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5274 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 758 3035.4880 0.3060 73.5870 4.0046
MXNDArrayCreateEx 879 30.3710 0.0310 0.0640 0.0346
MXNDArrayWaitAll 759 16467.4551 5.6130 25.0610 21.6963
MXNDArraySyncCopyToCPU 1638 12985.2617 0.0600 24.1980 7.9275
MXNDArrayGetDType 1638 0.5930 0.0000 0.0010 0.0004
MXNDArrayGetShape 4155 1.9590 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1758 15.4340 0.0070 0.0280 0.0088
MXNDArrayFree 19617 201.6810 0.0000 0.9140 0.0103
MXNet C API Calls 87813 575.1700 487.3580 575.1700 43.9060
MXNet C API Concurrency 175625 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1518 0.7140 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7590 3.9420 0.0000 0.0230 0.0005
MXNDArraySetGradState 7590 3.8720 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1518 0.6530 0.0000 0.0010 0.0004
MXAutogradBackwardEx 759 420.5030 0.5250 0.9450 0.5540
MXNDArrayGetContext 12186 5.3340 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25449 653.0250 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3036 7426.1489 0.7010 3.7500 2.4460
FullyConnected 5274 508.6640 0.0080 0.3430 0.0964
argmax 1758 34.8100 0.0050 0.0480 0.0198
_backward_pick 1518 24.1280 0.0130 0.0320 0.0159
_backward_mul_scalar 1518 6.6490 0.0030 0.0220 0.0044
DeleteVariable 65040 465.1780 0.0010 0.9010 0.0072
Flatten 1758 73.8430 0.0080 0.1120 0.0420
Activation 7032 326.8950 0.0030 1.9440 0.0465
SetValueOp 1518 9.8480 0.0050 0.0180 0.0065
pick 1518 11.9700 0.0060 0.0280 0.0079
Pooling 3516 4908.2832 0.0640 3.1720 1.3960
Convolution 3516 20017.1172 0.2260 13.2050 5.6932
CopyCPU2CPU 1758 10.1730 0.0030 0.0360 0.0058
log_softmax 1518 47.2770 0.0150 0.0550 0.0311
_backward_copy 1518 78.4980 0.0220 0.0760 0.0517
_backward_Convolution 3036 28083.7227 2.6240 12.5610 9.2502
_mul_scalar 1518 5.4280 0.0020 0.0230 0.0036
_backward_log_softmax 1518 25.9090 0.0090 0.0400 0.0171
mean 4794 91.8460 0.0080 0.0730 0.0192
_backward_mean 1518 16.7030 0.0080 0.1030 0.0110
_backward_FullyConnected 4554 872.3960 0.0280 0.5940 0.1916
WaitForVar 3276 29.1130 0.0060 0.0280 0.0089
broadcast_equal 1758 8.8240 0.0030 0.0110 0.0050
sgd_update 15180 151.0040 0.0020 0.0790 0.0099
_backward_Activation 6072 448.9980 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59840 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5280 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 759 3039.4199 0.3060 73.5870 4.0045
MXNDArrayCreateEx 880 30.4070 0.0310 0.0640 0.0346
MXNDArrayWaitAll 760 16489.1719 5.6130 25.0610 21.6963
MXNDArraySyncCopyToCPU 1640 12999.9092 0.0600 24.1980 7.9268
MXNDArrayGetDType 1640 0.5930 0.0000 0.0010 0.0004
MXNDArrayGetShape 4160 1.9610 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1760 15.4530 0.0070 0.0280 0.0088
MXNDArrayFree 19640 201.9230 0.0000 0.9140 0.0103
MXNet C API Calls 87920 575.2770 487.3580 575.2770 43.9595
MXNet C API Concurrency 175839 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1520 0.7160 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7600 3.9460 0.0000 0.0230 0.0005
MXNDArraySetGradState 7600 3.8770 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1520 0.6530 0.0000 0.0010 0.0004
MXAutogradBackwardEx 760 421.0580 0.5250 0.9450 0.5540
MXNDArrayGetContext 12200 5.3400 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25480 653.8790 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3040 7435.9722 0.7010 3.7500 2.4460
FullyConnected 5280 509.4220 0.0080 0.3430 0.0965
argmax 1760 34.8550 0.0050 0.0480 0.0198
_backward_pick 1520 24.1610 0.0130 0.0320 0.0159
_backward_mul_scalar 1520 6.6570 0.0030 0.0220 0.0044
DeleteVariable 65120 465.7210 0.0010 0.9010 0.0072
Flatten 1760 73.9220 0.0080 0.1120 0.0420
Activation 7040 327.2730 0.0030 1.9440 0.0465
SetValueOp 1520 9.8590 0.0050 0.0180 0.0065
pick 1520 11.9860 0.0060 0.0280 0.0079
Pooling 3520 4913.6768 0.0640 3.1720 1.3959
Convolution 3520 20039.3809 0.2260 13.2050 5.6930
CopyCPU2CPU 1760 10.1830 0.0030 0.0360 0.0058
log_softmax 1520 47.3510 0.0150 0.0550 0.0312
_backward_copy 1520 78.6010 0.0220 0.0760 0.0517
_backward_Convolution 3040 28120.4609 2.6240 12.5610 9.2502
_mul_scalar 1520 5.4350 0.0020 0.0230 0.0036
_backward_log_softmax 1520 25.9510 0.0090 0.0400 0.0171
mean 4800 91.9540 0.0080 0.0730 0.0192
_backward_mean 1520 16.7250 0.0080 0.1030 0.0110
_backward_FullyConnected 4560 873.8190 0.0280 0.5940 0.1916
WaitForVar 3280 29.1490 0.0060 0.0280 0.0089
broadcast_equal 1760 8.8350 0.0030 0.0110 0.0050
sgd_update 15200 151.2770 0.0020 0.0790 0.0100
_backward_Activation 6080 449.5910 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59914 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5286 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 760 3043.3450 0.3060 73.5870 4.0044
MXNDArrayCreateEx 881 30.4430 0.0310 0.0640 0.0346
MXNDArrayWaitAll 761 16511.1074 5.6130 25.0610 21.6966
MXNDArraySyncCopyToCPU 1642 13014.1064 0.0600 24.1980 7.9258
MXNDArrayGetDType 1642 0.5930 0.0000 0.0010 0.0004
MXNDArrayGetShape 4165 1.9640 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1762 15.4720 0.0070 0.0280 0.0088
MXNDArrayFree 19663 202.1640 0.0000 0.9140 0.0103
MXNet C API Calls 88027 575.3840 487.3580 575.3840 44.0130
MXNet C API Concurrency 176053 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1522 0.7170 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7610 3.9530 0.0000 0.0230 0.0005
MXNDArraySetGradState 7610 3.8820 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1522 0.6530 0.0000 0.0010 0.0004
MXAutogradBackwardEx 761 421.6110 0.5250 0.9450 0.5540
MXNDArrayGetContext 12214 5.3450 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25511 654.7100 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3044 7445.7671 0.7010 3.7500 2.4460
FullyConnected 5286 509.9500 0.0080 0.3430 0.0965
argmax 1762 34.8930 0.0050 0.0480 0.0198
_backward_pick 1522 24.1920 0.0130 0.0320 0.0159
_backward_mul_scalar 1522 6.6660 0.0030 0.0220 0.0044
DeleteVariable 65200 466.2700 0.0010 0.9010 0.0072
Flatten 1762 73.9990 0.0080 0.1120 0.0420
Activation 7048 327.6410 0.0030 1.9440 0.0465
SetValueOp 1522 9.8720 0.0050 0.0180 0.0065
pick 1522 12.0010 0.0060 0.0280 0.0079
Pooling 3524 4919.0601 0.0640 3.1720 1.3959
Convolution 3524 20061.5664 0.2260 13.2050 5.6928
CopyCPU2CPU 1762 10.1920 0.0030 0.0360 0.0058
log_softmax 1522 47.4090 0.0150 0.0550 0.0311
_backward_copy 1522 78.7080 0.0220 0.0760 0.0517
_backward_Convolution 3044 28157.5430 2.6240 12.5610 9.2502
_mul_scalar 1522 5.4420 0.0020 0.0230 0.0036
_backward_log_softmax 1522 25.9830 0.0090 0.0400 0.0171
mean 4806 92.0700 0.0080 0.0730 0.0192
_backward_mean 1522 16.7480 0.0080 0.1030 0.0110
_backward_FullyConnected 4566 874.8900 0.0280 0.5940 0.1916
WaitForVar 3284 29.1860 0.0060 0.0280 0.0089
broadcast_equal 1762 8.8450 0.0030 0.0110 0.0050
sgd_update 15220 151.4300 0.0020 0.0790 0.0099
_backward_Activation 6088 450.2340 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 59988 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5292 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 761 3047.2729 0.3060 73.5870 4.0043
MXNDArrayCreateEx 882 30.4790 0.0310 0.0640 0.0346
MXNDArrayWaitAll 762 16532.8848 5.6130 25.0610 21.6967
MXNDArraySyncCopyToCPU 1644 13028.3994 0.0600 24.1980 7.9248
MXNDArrayGetDType 1644 0.5940 0.0000 0.0010 0.0004
MXNDArrayGetShape 4170 1.9660 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1764 15.4910 0.0070 0.0280 0.0088
MXNDArrayFree 19686 202.4030 0.0000 0.9140 0.0103
MXNet C API Calls 88134 575.4910 487.3580 575.4910 44.0665
MXNet C API Concurrency 176267 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1524 0.7180 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7620 3.9580 0.0000 0.0230 0.0005
MXNDArraySetGradState 7620 3.8860 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1524 0.6530 0.0000 0.0010 0.0004
MXAutogradBackwardEx 762 422.1590 0.5250 0.9450 0.5540
MXNDArrayGetContext 12228 5.3520 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25542 655.5400 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3048 7455.3560 0.7010 3.7500 2.4460
FullyConnected 5292 510.4810 0.0080 0.3430 0.0965
argmax 1764 34.9300 0.0050 0.0480 0.0198
_backward_pick 1524 24.2220 0.0130 0.0320 0.0159
_backward_mul_scalar 1524 6.6750 0.0030 0.0220 0.0044
DeleteVariable 65280 466.8250 0.0010 0.9010 0.0072
Flatten 1764 74.0770 0.0080 0.1120 0.0420
Activation 7056 328.0080 0.0030 1.9440 0.0465
SetValueOp 1524 9.8860 0.0050 0.0180 0.0065
pick 1524 12.0170 0.0060 0.0280 0.0079
Pooling 3528 4924.4570 0.0640 3.1720 1.3958
Convolution 3528 20083.8262 0.2260 13.2050 5.6927
CopyCPU2CPU 1764 10.2010 0.0030 0.0360 0.0058
log_softmax 1524 47.4690 0.0150 0.0550 0.0311
_backward_copy 1524 78.8150 0.0220 0.0760 0.0517
_backward_Convolution 3048 28194.5859 2.6240 12.5610 9.2502
_mul_scalar 1524 5.4500 0.0020 0.0230 0.0036
_backward_log_softmax 1524 26.0160 0.0090 0.0400 0.0171
mean 4812 92.1830 0.0080 0.0730 0.0192
_backward_mean 1524 16.7700 0.0080 0.1030 0.0110
_backward_FullyConnected 4572 875.9630 0.0280 0.5940 0.1916
WaitForVar 3288 29.2220 0.0060 0.0280 0.0089
broadcast_equal 1764 8.8540 0.0030 0.0110 0.0050
sgd_update 15240 151.5800 0.0020 0.0790 0.0099
_backward_Activation 6096 450.8830 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60062 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5298 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 762 3051.1870 0.3060 73.5870 4.0042
MXNDArrayCreateEx 883 30.5150 0.0310 0.0640 0.0346
MXNDArrayWaitAll 763 16554.5410 5.6130 25.0610 21.6966
MXNDArraySyncCopyToCPU 1646 13042.6543 0.0600 24.1980 7.9238
MXNDArrayGetDType 1646 0.5940 0.0000 0.0010 0.0004
MXNDArrayGetShape 4175 1.9680 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1766 15.5090 0.0070 0.0280 0.0088
MXNDArrayFree 19709 202.6340 0.0000 0.9140 0.0103
MXNet C API Calls 88241 575.5980 487.3580 575.5980 44.1200
MXNet C API Concurrency 176481 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1526 0.7180 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7630 3.9610 0.0000 0.0230 0.0005
MXNDArraySetGradState 7630 3.8880 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1526 0.6530 0.0000 0.0010 0.0004
MXAutogradBackwardEx 763 422.7080 0.5250 0.9450 0.5540
MXNDArrayGetContext 12242 5.3580 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25573 656.3590 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3052 7465.1118 0.7010 3.7500 2.4460
FullyConnected 5298 511.0060 0.0080 0.3430 0.0965
argmax 1766 34.9680 0.0050 0.0480 0.0198
_backward_pick 1526 24.2540 0.0130 0.0320 0.0159
_backward_mul_scalar 1526 6.6830 0.0030 0.0220 0.0044
DeleteVariable 65360 467.3580 0.0010 0.9010 0.0072
Flatten 1766 74.1560 0.0080 0.1120 0.0420
Activation 7064 328.3770 0.0030 1.9440 0.0465
SetValueOp 1526 9.8990 0.0050 0.0180 0.0065
pick 1526 12.0320 0.0060 0.0280 0.0079
Pooling 3532 4929.8140 0.0640 3.1720 1.3958
Convolution 3532 20106.0332 0.2260 13.2050 5.6925
CopyCPU2CPU 1766 10.2110 0.0030 0.0360 0.0058
log_softmax 1526 47.5280 0.0150 0.0550 0.0311
_backward_copy 1526 78.9200 0.0220 0.0760 0.0517
_backward_Convolution 3052 28231.4199 2.6240 12.5610 9.2501
_mul_scalar 1526 5.4570 0.0020 0.0230 0.0036
_backward_log_softmax 1526 26.0470 0.0090 0.0400 0.0171
mean 4818 92.3400 0.0080 0.0730 0.0192
_backward_mean 1526 16.7920 0.0080 0.1030 0.0110
_backward_FullyConnected 4578 877.0360 0.0280 0.5940 0.1916
WaitForVar 3292 29.2570 0.0060 0.0280 0.0089
broadcast_equal 1766 8.8630 0.0030 0.0110 0.0050
sgd_update 15260 151.7810 0.0020 0.0790 0.0099
_backward_Activation 6104 451.4290 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60136 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5304 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 763 3055.1111 0.3060 73.5870 4.0041
MXNDArrayCreateEx 884 30.5520 0.0310 0.0640 0.0346
MXNDArrayWaitAll 764 16576.2461 5.6130 25.0610 21.6967
MXNDArraySyncCopyToCPU 1648 13057.2188 0.0600 24.1980 7.9231
MXNDArrayGetDType 1648 0.5950 0.0000 0.0010 0.0004
MXNDArrayGetShape 4180 1.9700 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1768 15.5280 0.0070 0.0280 0.0088
MXNDArrayFree 19732 202.8700 0.0000 0.9140 0.0103
MXNet C API Calls 88348 575.7050 487.3580 575.7050 44.1735
MXNet C API Concurrency 176695 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1528 0.7180 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7640 3.9660 0.0000 0.0230 0.0005
MXNDArraySetGradState 7640 3.8950 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1528 0.6530 0.0000 0.0010 0.0004
MXAutogradBackwardEx 764 423.2590 0.5250 0.9450 0.5540
MXNDArrayGetContext 12256 5.3650 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25604 657.2160 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3056 7474.8848 0.7010 3.7500 2.4460
FullyConnected 5304 511.7340 0.0080 0.3430 0.0965
argmax 1768 35.0140 0.0050 0.0480 0.0198
_backward_pick 1528 24.2870 0.0130 0.0320 0.0159
_backward_mul_scalar 1528 6.6910 0.0030 0.0220 0.0044
DeleteVariable 65440 467.8870 0.0010 0.9010 0.0071
Flatten 1768 74.2350 0.0080 0.1120 0.0420
Activation 7072 328.7600 0.0030 1.9440 0.0465
SetValueOp 1528 9.9130 0.0050 0.0180 0.0065
pick 1528 12.0480 0.0060 0.0280 0.0079
Pooling 3536 4935.1748 0.0640 3.1720 1.3957
Convolution 3536 20128.2598 0.2260 13.2050 5.6924
CopyCPU2CPU 1768 10.2210 0.0030 0.0360 0.0058
log_softmax 1528 47.5990 0.0150 0.0550 0.0312
_backward_copy 1528 79.0230 0.0220 0.0760 0.0517
_backward_Convolution 3056 28268.1465 2.6240 12.5610 9.2500
_mul_scalar 1528 5.4640 0.0020 0.0230 0.0036
_backward_log_softmax 1528 26.0900 0.0090 0.0400 0.0171
mean 4824 92.4500 0.0080 0.0730 0.0192
_backward_mean 1528 16.8130 0.0080 0.1030 0.0110
_backward_FullyConnected 4584 878.4430 0.0280 0.5940 0.1916
WaitForVar 3296 29.2920 0.0060 0.0280 0.0089
broadcast_equal 1768 8.8730 0.0030 0.0110 0.0050
sgd_update 15280 152.0600 0.0020 0.0790 0.0100
_backward_Activation 6112 452.0340 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60210 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5310 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 764 3059.0400 0.3060 73.5870 4.0040
MXNDArrayCreateEx 885 30.6120 0.0310 0.0640 0.0346
MXNDArrayWaitAll 765 16598.0918 5.6130 25.0610 21.6969
MXNDArraySyncCopyToCPU 1650 13071.4160 0.0600 24.1980 7.9221
MXNDArrayGetDType 1650 0.5950 0.0000 0.0010 0.0004
MXNDArrayGetShape 4185 1.9740 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1770 15.5450 0.0070 0.0280 0.0088
MXNDArrayFree 19755 203.1010 0.0000 0.9140 0.0103
MXNet C API Calls 88455 575.8120 487.3580 575.8120 44.2270
MXNet C API Concurrency 176909 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1530 0.7190 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7650 3.9720 0.0000 0.0230 0.0005
MXNDArraySetGradState 7650 3.9020 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1530 0.6550 0.0000 0.0010 0.0004
MXAutogradBackwardEx 765 423.8070 0.5250 0.9450 0.5540
MXNDArrayGetContext 12270 5.3720 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25635 658.0420 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3060 7484.5229 0.7010 3.7500 2.4459
FullyConnected 5310 512.2560 0.0080 0.3430 0.0965
argmax 1770 35.0540 0.0050 0.0480 0.0198
_backward_pick 1530 24.3190 0.0130 0.0320 0.0159
_backward_mul_scalar 1530 6.7000 0.0030 0.0220 0.0044
DeleteVariable 65520 468.4180 0.0010 0.9010 0.0071
Flatten 1770 74.3130 0.0080 0.1120 0.0420
Activation 7080 329.1290 0.0030 1.9440 0.0465
SetValueOp 1530 9.9260 0.0050 0.0180 0.0065
pick 1530 12.0630 0.0060 0.0280 0.0079
Pooling 3540 4940.4990 0.0640 3.1720 1.3956
Convolution 3540 20150.4492 0.2260 13.2050 5.6922
CopyCPU2CPU 1770 10.2310 0.0030 0.0360 0.0058
log_softmax 1530 47.6590 0.0150 0.0550 0.0311
_backward_copy 1530 79.1270 0.0220 0.0760 0.0517
_backward_Convolution 3060 28305.2168 2.6240 12.5610 9.2501
_mul_scalar 1530 5.4710 0.0020 0.0230 0.0036
_backward_log_softmax 1530 26.1210 0.0090 0.0400 0.0171
mean 4830 92.5690 0.0080 0.0730 0.0192
_backward_mean 1530 16.8340 0.0080 0.1030 0.0110
_backward_FullyConnected 4590 879.5510 0.0280 0.5940 0.1916
WaitForVar 3300 29.3270 0.0060 0.0280 0.0089
broadcast_equal 1770 8.8820 0.0030 0.0110 0.0050
sgd_update 15300 152.2650 0.0020 0.0790 0.0100
_backward_Activation 6120 452.6140 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60284 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5316 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 765 3062.9629 0.3060 73.5870 4.0039
MXNDArrayCreateEx 886 30.6480 0.0310 0.0640 0.0346
MXNDArrayWaitAll 766 16620.0000 5.6130 25.0610 21.6971
MXNDArraySyncCopyToCPU 1652 13085.7363 0.0600 24.1980 7.9211
MXNDArrayGetDType 1652 0.5960 0.0000 0.0010 0.0004
MXNDArrayGetShape 4190 1.9740 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1772 15.5630 0.0070 0.0280 0.0088
MXNDArrayFree 19778 203.3600 0.0000 0.9140 0.0103
MXNet C API Calls 88562 575.9190 487.3580 575.9190 44.2805
MXNet C API Concurrency 177123 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1532 0.7190 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7660 3.9750 0.0000 0.0230 0.0005
MXNDArraySetGradState 7660 3.9060 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1532 0.6550 0.0000 0.0010 0.0004
MXAutogradBackwardEx 766 424.3530 0.5250 0.9450 0.5540
MXNDArrayGetContext 12284 5.3740 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25666 658.8710 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3064 7494.3682 0.7010 3.7500 2.4459
FullyConnected 5316 512.8020 0.0080 0.3430 0.0965
argmax 1772 35.0930 0.0050 0.0480 0.0198
_backward_pick 1532 24.3500 0.0130 0.0320 0.0159
_backward_mul_scalar 1532 6.7080 0.0030 0.0220 0.0044
DeleteVariable 65600 468.9400 0.0010 0.9010 0.0071
Flatten 1772 74.3910 0.0080 0.1120 0.0420
Activation 7088 329.5000 0.0030 1.9440 0.0465
SetValueOp 1532 9.9390 0.0050 0.0180 0.0065
pick 1532 12.0780 0.0060 0.0280 0.0079
Pooling 3544 4945.8979 0.0640 3.1720 1.3956
Convolution 3544 20172.6895 0.2260 13.2050 5.6921
CopyCPU2CPU 1772 10.2410 0.0030 0.0360 0.0058
log_softmax 1532 47.7180 0.0150 0.0550 0.0311
_backward_copy 1532 79.2320 0.0220 0.0760 0.0517
_backward_Convolution 3064 28342.3027 2.6240 12.5610 9.2501
_mul_scalar 1532 5.4780 0.0020 0.0230 0.0036
_backward_log_softmax 1532 26.1530 0.0090 0.0400 0.0171
mean 4836 92.6890 0.0080 0.0730 0.0192
_backward_mean 1532 16.8550 0.0080 0.1030 0.0110
_backward_FullyConnected 4596 880.6250 0.0280 0.5940 0.1916
WaitForVar 3304 29.3620 0.0060 0.0280 0.0089
broadcast_equal 1772 8.8910 0.0030 0.0110 0.0050
sgd_update 15320 152.4690 0.0020 0.0790 0.0100
_backward_Activation 6128 453.1860 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60358 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5322 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 766 3066.8840 0.3060 73.5870 4.0038
MXNDArrayCreateEx 887 30.6840 0.0310 0.0640 0.0346
MXNDArrayWaitAll 767 16641.6992 5.6130 25.0610 21.6971
MXNDArraySyncCopyToCPU 1654 13099.9805 0.0600 24.1980 7.9202
MXNDArrayGetDType 1654 0.5960 0.0000 0.0010 0.0004
MXNDArrayGetShape 4195 1.9750 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1774 15.5820 0.0070 0.0280 0.0088
MXNDArrayFree 19801 203.5950 0.0000 0.9140 0.0103
MXNet C API Calls 88669 576.0260 487.3580 576.0260 44.3340
MXNet C API Concurrency 177337 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1534 0.7200 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7670 3.9800 0.0000 0.0230 0.0005
MXNDArraySetGradState 7670 3.9110 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1534 0.6570 0.0000 0.0010 0.0004
MXAutogradBackwardEx 767 424.9010 0.5250 0.9450 0.5540
MXNDArrayGetContext 12298 5.3810 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25697 659.6900 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3068 7504.0991 0.7010 3.7500 2.4459
FullyConnected 5322 513.3240 0.0080 0.3430 0.0965
argmax 1774 35.1360 0.0050 0.0480 0.0198
_backward_pick 1534 24.3810 0.0130 0.0320 0.0159
_backward_mul_scalar 1534 6.7170 0.0030 0.0220 0.0044
DeleteVariable 65680 469.4900 0.0010 0.9010 0.0071
Flatten 1774 74.4690 0.0080 0.1120 0.0420
Activation 7096 329.8710 0.0030 1.9440 0.0465
SetValueOp 1534 9.9520 0.0050 0.0180 0.0065
pick 1534 12.0930 0.0060 0.0280 0.0079
Pooling 3548 4951.2231 0.0640 3.1720 1.3955
Convolution 3548 20194.8809 0.2260 13.2050 5.6919
CopyCPU2CPU 1774 10.2510 0.0030 0.0360 0.0058
log_softmax 1534 47.7780 0.0150 0.0550 0.0311
_backward_copy 1534 79.3380 0.0220 0.0760 0.0517
_backward_Convolution 3068 28379.0312 2.6240 12.5610 9.2500
_mul_scalar 1534 5.4860 0.0020 0.0230 0.0036
_backward_log_softmax 1534 26.1840 0.0090 0.0400 0.0171
mean 4842 92.8080 0.0080 0.0730 0.0192
_backward_mean 1534 16.8760 0.0080 0.1030 0.0110
_backward_FullyConnected 4602 881.6960 0.0280 0.5940 0.1916
WaitForVar 3308 29.3980 0.0060 0.0280 0.0089
broadcast_equal 1774 8.9010 0.0030 0.0110 0.0050
sgd_update 15340 152.6750 0.0020 0.0790 0.0100
_backward_Activation 6136 453.7310 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60432 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5328 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 767 3070.7991 0.3060 73.5870 4.0036
MXNDArrayCreateEx 888 30.7200 0.0310 0.0640 0.0346
MXNDArrayWaitAll 768 16663.5059 5.6130 25.0610 21.6973
MXNDArraySyncCopyToCPU 1656 13114.5391 0.0600 24.1980 7.9194
MXNDArrayGetDType 1656 0.5970 0.0000 0.0010 0.0004
MXNDArrayGetShape 4200 1.9780 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1776 15.5990 0.0070 0.0280 0.0088
MXNDArrayFree 19824 203.8300 0.0000 0.9140 0.0103
MXNet C API Calls 88776 576.1330 487.3580 576.1330 44.3875
MXNet C API Concurrency 177551 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1536 0.7210 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7680 3.9850 0.0000 0.0230 0.0005
MXNDArraySetGradState 7680 3.9160 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1536 0.6580 0.0000 0.0010 0.0004
MXAutogradBackwardEx 768 425.4730 0.5250 0.9450 0.5540
MXNDArrayGetContext 12312 5.3870 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25728 660.5070 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3072 7513.8809 0.7010 3.7500 2.4459
FullyConnected 5328 514.0800 0.0080 0.3430 0.0965
argmax 1776 35.1800 0.0050 0.0480 0.0198
_backward_pick 1536 24.4140 0.0130 0.0320 0.0159
_backward_mul_scalar 1536 6.7260 0.0030 0.0220 0.0044
DeleteVariable 65760 470.0210 0.0010 0.9010 0.0071
Flatten 1776 74.5490 0.0080 0.1120 0.0420
Activation 7104 330.2500 0.0030 1.9440 0.0465
SetValueOp 1536 9.9650 0.0050 0.0180 0.0065
pick 1536 12.1090 0.0060 0.0280 0.0079
Pooling 3552 4956.5708 0.0640 3.1720 1.3954
Convolution 3552 20217.1035 0.2260 13.2050 5.6918
CopyCPU2CPU 1776 10.2600 0.0030 0.0360 0.0058
log_softmax 1536 47.8500 0.0150 0.0550 0.0312
_backward_copy 1536 79.4400 0.0220 0.0760 0.0517
_backward_Convolution 3072 28415.8711 2.6240 12.5610 9.2500
_mul_scalar 1536 5.4930 0.0020 0.0230 0.0036
_backward_log_softmax 1536 26.2260 0.0090 0.0400 0.0171
mean 4848 92.9180 0.0080 0.0730 0.0192
_backward_mean 1536 16.8980 0.0080 0.1030 0.0110
_backward_FullyConnected 4608 883.1030 0.0280 0.5940 0.1916
WaitForVar 3312 29.4330 0.0060 0.0280 0.0089
broadcast_equal 1776 8.9100 0.0030 0.0110 0.0050
sgd_update 15360 152.9530 0.0020 0.0790 0.0100
_backward_Activation 6144 454.3050 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60506 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5334 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 768 3074.7229 0.3060 73.5870 4.0035
MXNDArrayCreateEx 889 30.7550 0.0310 0.0640 0.0346
MXNDArrayWaitAll 769 16685.3691 5.6130 25.0610 21.6975
MXNDArraySyncCopyToCPU 1658 13128.7930 0.0600 24.1980 7.9185
MXNDArrayGetDType 1658 0.5970 0.0000 0.0010 0.0004
MXNDArrayGetShape 4205 1.9810 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1778 15.6170 0.0070 0.0280 0.0088
MXNDArrayFree 19847 204.0680 0.0000 0.9140 0.0103
MXNet C API Calls 88883 576.2400 487.3580 576.2400 44.4410
MXNet C API Concurrency 177765 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1538 0.7220 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7690 3.9930 0.0000 0.0230 0.0005
MXNDArraySetGradState 7690 3.9230 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1538 0.6590 0.0000 0.0010 0.0004
MXAutogradBackwardEx 769 426.0240 0.5250 0.9450 0.5540
MXNDArrayGetContext 12326 5.3950 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25759 661.3500 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3076 7523.6372 0.7010 3.7500 2.4459
FullyConnected 5334 514.6050 0.0080 0.3430 0.0965
argmax 1778 35.2180 0.0050 0.0480 0.0198
_backward_pick 1538 24.4460 0.0130 0.0320 0.0159
_backward_mul_scalar 1538 6.7350 0.0030 0.0220 0.0044
DeleteVariable 65840 470.5640 0.0010 0.9010 0.0071
Flatten 1778 74.6280 0.0080 0.1120 0.0420
Activation 7112 330.6190 0.0030 1.9440 0.0465
SetValueOp 1538 9.9780 0.0050 0.0180 0.0065
pick 1538 12.1240 0.0060 0.0280 0.0079
Pooling 3556 4961.9419 0.0640 3.1720 1.3954
Convolution 3556 20239.2949 0.2260 13.2050 5.6916
CopyCPU2CPU 1778 10.2700 0.0030 0.0360 0.0058
log_softmax 1538 47.9100 0.0150 0.0550 0.0312
_backward_copy 1538 79.5470 0.0220 0.0760 0.0517
_backward_Convolution 3076 28452.9219 2.6240 12.5610 9.2500
_mul_scalar 1538 5.5000 0.0020 0.0230 0.0036
_backward_log_softmax 1538 26.2590 0.0090 0.0400 0.0171
mean 4854 93.0400 0.0080 0.0730 0.0192
_backward_mean 1538 16.9200 0.0080 0.1030 0.0110
_backward_FullyConnected 4614 884.1730 0.0280 0.5940 0.1916
WaitForVar 3316 29.4680 0.0060 0.0280 0.0089
broadcast_equal 1778 8.9190 0.0030 0.0110 0.0050
sgd_update 15380 153.1590 0.0020 0.0790 0.0100
_backward_Activation 6152 454.8740 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60580 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5340 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 769 3078.6450 0.3060 73.5870 4.0034
MXNDArrayCreateEx 890 30.7910 0.0310 0.0640 0.0346
MXNDArrayWaitAll 770 16707.2676 5.6130 25.0610 21.6978
MXNDArraySyncCopyToCPU 1660 13143.0283 0.0600 24.1980 7.9175
MXNDArrayGetDType 1660 0.5970 0.0000 0.0010 0.0004
MXNDArrayGetShape 4210 1.9850 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1780 15.6350 0.0070 0.0280 0.0088
MXNDArrayFree 19870 204.3030 0.0000 0.9140 0.0103
MXNet C API Calls 88990 576.3470 487.3580 576.3470 44.4945
MXNet C API Concurrency 177979 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1540 0.7230 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7700 3.9990 0.0000 0.0230 0.0005
MXNDArraySetGradState 7700 3.9280 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1540 0.6600 0.0000 0.0010 0.0004
MXAutogradBackwardEx 770 426.5730 0.5250 0.9450 0.5540
MXNDArrayGetContext 12340 5.4020 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25790 662.1790 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3080 7533.3389 0.7010 3.7500 2.4459
FullyConnected 5340 515.1270 0.0080 0.3430 0.0965
argmax 1780 35.2560 0.0050 0.0480 0.0198
_backward_pick 1540 24.4770 0.0130 0.0320 0.0159
_backward_mul_scalar 1540 6.7430 0.0030 0.0220 0.0044
DeleteVariable 65920 471.1100 0.0010 0.9010 0.0071
Flatten 1780 74.7070 0.0080 0.1120 0.0420
Activation 7120 331.0260 0.0030 1.9440 0.0465
SetValueOp 1540 9.9910 0.0050 0.0180 0.0065
pick 1540 12.1390 0.0060 0.0280 0.0079
Pooling 3560 4967.3110 0.0640 3.1720 1.3953
Convolution 3560 20261.5391 0.2260 13.2050 5.6914
CopyCPU2CPU 1780 10.2800 0.0030 0.0360 0.0058
log_softmax 1540 47.9680 0.0150 0.0550 0.0311
_backward_copy 1540 79.6530 0.0220 0.0760 0.0517
_backward_Convolution 3080 28490.0254 2.6240 12.5610 9.2500
_mul_scalar 1540 5.5070 0.0020 0.0230 0.0036
_backward_log_softmax 1540 26.2910 0.0090 0.0400 0.0171
mean 4860 93.1580 0.0080 0.0730 0.0192
_backward_mean 1540 16.9420 0.0080 0.1030 0.0110
_backward_FullyConnected 4620 885.2340 0.0280 0.5940 0.1916
WaitForVar 3320 29.5040 0.0060 0.0280 0.0089
broadcast_equal 1780 8.9290 0.0030 0.0110 0.0050
sgd_update 15400 153.3650 0.0020 0.0790 0.0100
_backward_Activation 6160 455.4660 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60654 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5346 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 770 3082.5581 0.3060 73.5870 4.0033
MXNDArrayCreateEx 891 30.8280 0.0310 0.0640 0.0346
MXNDArrayWaitAll 771 16728.9746 5.6130 25.0610 21.6978
MXNDArraySyncCopyToCPU 1662 13157.2539 0.0600 24.1980 7.9165
MXNDArrayGetDType 1662 0.5980 0.0000 0.0010 0.0004
MXNDArrayGetShape 4215 1.9870 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1782 15.6540 0.0070 0.0280 0.0088
MXNDArrayFree 19893 204.5370 0.0000 0.9140 0.0103
MXNet C API Calls 89097 576.4540 487.3580 576.4540 44.5480
MXNet C API Concurrency 178193 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1542 0.7240 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7710 4.0040 0.0000 0.0230 0.0005
MXNDArraySetGradState 7710 3.9340 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1542 0.6610 0.0000 0.0010 0.0004
MXAutogradBackwardEx 771 427.1210 0.5250 0.9450 0.5540
MXNDArrayGetContext 12354 5.4110 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25821 663.0270 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3084 7543.0801 0.7010 3.7500 2.4459
FullyConnected 5346 515.6470 0.0080 0.3430 0.0965
argmax 1782 35.2940 0.0050 0.0480 0.0198
_backward_pick 1542 24.5070 0.0130 0.0320 0.0159
_backward_mul_scalar 1542 6.7520 0.0030 0.0220 0.0044
DeleteVariable 66000 471.6250 0.0010 0.9010 0.0071
Flatten 1782 74.7850 0.0080 0.1120 0.0420
Activation 7128 331.3940 0.0030 1.9440 0.0465
SetValueOp 1542 10.0040 0.0050 0.0180 0.0065
pick 1542 12.1540 0.0060 0.0280 0.0079
Pooling 3564 4972.6548 0.0640 3.1720 1.3952
Convolution 3564 20283.7383 0.2260 13.2050 5.6913
CopyCPU2CPU 1782 10.2900 0.0030 0.0360 0.0058
log_softmax 1542 48.0290 0.0150 0.0550 0.0311
_backward_copy 1542 79.7580 0.0220 0.0760 0.0517
_backward_Convolution 3084 28526.7773 2.6240 12.5610 9.2499
_mul_scalar 1542 5.5140 0.0020 0.0230 0.0036
_backward_log_softmax 1542 26.3230 0.0090 0.0400 0.0171
mean 4866 93.2760 0.0080 0.0730 0.0192
_backward_mean 1542 16.9640 0.0080 0.1030 0.0110
_backward_FullyConnected 4626 886.3100 0.0280 0.5940 0.1916
WaitForVar 3324 29.5400 0.0060 0.0280 0.0089
broadcast_equal 1782 8.9390 0.0030 0.0110 0.0050
sgd_update 15420 153.5720 0.0020 0.0790 0.0100
_backward_Activation 6168 456.0370 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60728 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5352 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 771 3086.4900 0.3060 73.5870 4.0032
MXNDArrayCreateEx 892 30.8640 0.0310 0.0640 0.0346
MXNDArrayWaitAll 772 16750.7129 5.6130 25.0610 21.6978
MXNDArraySyncCopyToCPU 1664 13171.8408 0.0600 24.1980 7.9158
MXNDArrayGetDType 1664 0.5980 0.0000 0.0010 0.0004
MXNDArrayGetShape 4220 1.9890 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1784 15.6730 0.0070 0.0280 0.0088
MXNDArrayFree 19916 204.7710 0.0000 0.9140 0.0103
MXNet C API Calls 89204 576.5610 487.3580 576.5610 44.6015
MXNet C API Concurrency 178407 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1544 0.7250 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7720 4.0080 0.0000 0.0230 0.0005
MXNDArraySetGradState 7720 3.9390 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1544 0.6620 0.0000 0.0010 0.0004
MXAutogradBackwardEx 772 427.6720 0.5250 0.9450 0.5540
MXNDArrayGetContext 12368 5.4190 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25852 663.8490 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3088 7552.8071 0.7010 3.7500 2.4459
FullyConnected 5352 516.3770 0.0080 0.3430 0.0965
argmax 1784 35.3380 0.0050 0.0480 0.0198
_backward_pick 1544 24.5400 0.0130 0.0320 0.0159
_backward_mul_scalar 1544 6.7600 0.0030 0.0220 0.0044
DeleteVariable 66080 472.1760 0.0010 0.9010 0.0071
Flatten 1784 74.8650 0.0080 0.1120 0.0420
Activation 7136 331.7750 0.0030 1.9440 0.0465
SetValueOp 1544 10.0170 0.0050 0.0180 0.0065
pick 1544 12.1690 0.0060 0.0280 0.0079
Pooling 3568 4978.0112 0.0640 3.1720 1.3952
Convolution 3568 20305.9414 0.2260 13.2050 5.6911
CopyCPU2CPU 1784 10.2990 0.0030 0.0360 0.0058
log_softmax 1544 48.1010 0.0150 0.0550 0.0312
_backward_copy 1544 79.8590 0.0220 0.0760 0.0517
_backward_Convolution 3088 28563.5508 2.6240 12.5610 9.2499
_mul_scalar 1544 5.5220 0.0020 0.0230 0.0036
_backward_log_softmax 1544 26.3640 0.0090 0.0400 0.0171
mean 4872 93.3860 0.0080 0.0730 0.0192
_backward_mean 1544 16.9850 0.0080 0.1030 0.0110
_backward_FullyConnected 4632 887.7710 0.0280 0.5940 0.1917
WaitForVar 3328 29.5750 0.0060 0.0280 0.0089
broadcast_equal 1784 8.9480 0.0030 0.0110 0.0050
sgd_update 15440 153.8470 0.0020 0.0790 0.0100
_backward_Activation 6176 456.6470 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60802 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5358 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 772 3086.8091 0.3060 73.5870 3.9985
MXNDArrayCreateEx 893 30.8990 0.0310 0.0640 0.0346
MXNDArrayWaitAll 773 16772.5449 5.6130 25.0610 21.6980
MXNDArraySyncCopyToCPU 1666 13186.0430 0.0600 24.1980 7.9148
MXNDArrayGetDType 1666 0.5990 0.0000 0.0010 0.0004
MXNDArrayGetShape 4225 1.9910 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1786 15.6920 0.0070 0.0280 0.0088
MXNDArrayFree 19939 205.0130 0.0000 0.9140 0.0103
MXNet C API Calls 89311 576.6680 487.3580 576.6680 44.6550
MXNet C API Concurrency 178621 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1546 0.7250 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7730 4.0110 0.0000 0.0230 0.0005
MXNDArraySetGradState 7730 3.9440 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1546 0.6620 0.0000 0.0010 0.0004
MXAutogradBackwardEx 773 428.2230 0.5250 0.9450 0.5540
MXNDArrayGetContext 12382 5.4260 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25883 664.6990 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3092 7562.5132 0.7010 3.7500 2.4458
FullyConnected 5358 516.9010 0.0080 0.3430 0.0965
argmax 1786 35.3770 0.0050 0.0480 0.0198
_backward_pick 1546 24.5710 0.0130 0.0320 0.0159
_backward_mul_scalar 1546 6.7680 0.0030 0.0220 0.0044
DeleteVariable 66160 472.7150 0.0010 0.9010 0.0071
Flatten 1786 74.9460 0.0080 0.1120 0.0420
Activation 7144 332.1440 0.0030 1.9440 0.0465
SetValueOp 1546 10.0320 0.0050 0.0180 0.0065
pick 1546 12.1840 0.0060 0.0280 0.0079
Pooling 3572 4983.4072 0.0640 3.1720 1.3951
Convolution 3572 20328.1055 0.2260 13.2050 5.6910
CopyCPU2CPU 1786 10.3090 0.0030 0.0360 0.0058
log_softmax 1546 48.1620 0.0150 0.0550 0.0312
_backward_copy 1546 79.9650 0.0220 0.0760 0.0517
_backward_Convolution 3092 28600.5703 2.6240 12.5610 9.2499
_mul_scalar 1546 5.5290 0.0020 0.0230 0.0036
_backward_log_softmax 1546 26.3960 0.0090 0.0400 0.0171
mean 4878 93.4990 0.0080 0.0730 0.0192
_backward_mean 1546 17.0070 0.0080 0.1030 0.0110
_backward_FullyConnected 4638 888.8450 0.0280 0.5940 0.1916
WaitForVar 3332 29.6100 0.0060 0.0280 0.0089
broadcast_equal 1786 8.9580 0.0030 0.0110 0.0050
sgd_update 15460 154.0530 0.0020 0.0790 0.0100
_backward_Activation 6184 457.2430 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60876 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5364 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 773 3090.7319 0.3060 73.5870 3.9984
MXNDArrayCreateEx 894 30.9350 0.0310 0.0640 0.0346
MXNDArrayWaitAll 774 16794.4102 5.6130 25.0610 21.6982
MXNDArraySyncCopyToCPU 1668 13200.2949 0.0600 24.1980 7.9138
MXNDArrayGetDType 1668 0.6000 0.0000 0.0010 0.0004
MXNDArrayGetShape 4230 1.9920 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1788 15.7100 0.0070 0.0280 0.0088
MXNDArrayFree 19962 205.2460 0.0000 0.9140 0.0103
MXNet C API Calls 89418 576.7750 487.3580 576.7750 44.7085
MXNet C API Concurrency 178835 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1548 0.7260 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7740 4.0140 0.0000 0.0230 0.0005
MXNDArraySetGradState 7740 3.9490 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1548 0.6620 0.0000 0.0010 0.0004
MXAutogradBackwardEx 774 428.7770 0.5250 0.9450 0.5540
MXNDArrayGetContext 12396 5.4320 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25914 665.5240 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3096 7572.2651 0.7010 3.7500 2.4458
FullyConnected 5364 517.4250 0.0080 0.3430 0.0965
argmax 1788 35.4150 0.0050 0.0480 0.0198
_backward_pick 1548 24.6030 0.0130 0.0320 0.0159
_backward_mul_scalar 1548 6.7770 0.0030 0.0220 0.0044
DeleteVariable 66240 473.2630 0.0010 0.9010 0.0071
Flatten 1788 75.0270 0.0080 0.1120 0.0420
Activation 7152 332.5120 0.0030 1.9440 0.0465
SetValueOp 1548 10.0450 0.0050 0.0180 0.0065
pick 1548 12.1990 0.0060 0.0280 0.0079
Pooling 3576 4988.7319 0.0640 3.1720 1.3951
Convolution 3576 20350.3574 0.2260 13.2050 5.6908
CopyCPU2CPU 1788 10.3190 0.0030 0.0360 0.0058
log_softmax 1548 48.2210 0.0150 0.0550 0.0312
_backward_copy 1548 80.0700 0.0220 0.0760 0.0517
_backward_Convolution 3096 28637.5820 2.6240 12.5610 9.2499
_mul_scalar 1548 5.5360 0.0020 0.0230 0.0036
_backward_log_softmax 1548 26.4280 0.0090 0.0400 0.0171
mean 4884 93.6140 0.0080 0.0730 0.0192
_backward_mean 1548 17.0290 0.0080 0.1030 0.0110
_backward_FullyConnected 4644 889.9530 0.0280 0.5940 0.1916
WaitForVar 3336 29.6440 0.0060 0.0280 0.0089
broadcast_equal 1788 8.9680 0.0030 0.0110 0.0050
sgd_update 15480 154.2050 0.0020 0.0790 0.0100
_backward_Activation 6192 457.8800 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 60950 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5370 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 774 3094.6580 0.3060 73.5870 3.9983
MXNDArrayCreateEx 895 30.9710 0.0310 0.0640 0.0346
MXNDArrayWaitAll 775 16816.1270 5.6130 25.0610 21.6982
MXNDArraySyncCopyToCPU 1670 13214.5195 0.0600 24.1980 7.9129
MXNDArrayGetDType 1670 0.6010 0.0000 0.0010 0.0004
MXNDArrayGetShape 4235 1.9940 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1790 15.7290 0.0070 0.0280 0.0088
MXNDArrayFree 19985 205.4890 0.0000 0.9140 0.0103
MXNet C API Calls 89525 576.8820 487.3580 576.8820 44.7620
MXNet C API Concurrency 179049 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1550 0.7270 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7750 4.0190 0.0000 0.0230 0.0005
MXNDArraySetGradState 7750 3.9540 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1550 0.6640 0.0000 0.0010 0.0004
MXAutogradBackwardEx 775 429.3560 0.5250 0.9450 0.5540
MXNDArrayGetContext 12410 5.4410 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25945 666.3450 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3100 7581.9590 0.7010 3.7500 2.4458
FullyConnected 5370 517.9570 0.0080 0.3430 0.0965
argmax 1790 35.4530 0.0050 0.0480 0.0198
_backward_pick 1550 24.6340 0.0130 0.0320 0.0159
_backward_mul_scalar 1550 6.7850 0.0030 0.0220 0.0044
DeleteVariable 66320 473.8210 0.0010 0.9010 0.0071
Flatten 1790 75.1070 0.0080 0.1120 0.0420
Activation 7160 332.8810 0.0030 1.9440 0.0465
SetValueOp 1550 10.0580 0.0050 0.0180 0.0065
pick 1550 12.2140 0.0060 0.0280 0.0079
Pooling 3580 4994.1650 0.0640 3.1720 1.3950
Convolution 3580 20372.4961 0.2260 13.2050 5.6906
CopyCPU2CPU 1790 10.3280 0.0030 0.0360 0.0058
log_softmax 1550 48.2800 0.0150 0.0550 0.0311
_backward_copy 1550 80.1750 0.0220 0.0760 0.0517
_backward_Convolution 3100 28674.4062 2.6240 12.5610 9.2498
_mul_scalar 1550 5.5430 0.0020 0.0230 0.0036
_backward_log_softmax 1550 26.4600 0.0090 0.0400 0.0171
mean 4890 93.7320 0.0080 0.0730 0.0192
_backward_mean 1550 17.0510 0.0080 0.1030 0.0110
_backward_FullyConnected 4650 891.0210 0.0280 0.5940 0.1916
WaitForVar 3340 29.6800 0.0060 0.0280 0.0089
broadcast_equal 1790 8.9770 0.0030 0.0110 0.0050
sgd_update 15500 154.4060 0.0020 0.0790 0.0100
_backward_Activation 6200 458.4270 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61024 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5376 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 775 3098.5801 0.3060 73.5870 3.9982
MXNDArrayCreateEx 896 31.0070 0.0310 0.0640 0.0346
MXNDArrayWaitAll 776 16837.8633 5.6130 25.0610 21.6983
MXNDArraySyncCopyToCPU 1672 13229.1006 0.0600 24.1980 7.9121
MXNDArrayGetDType 1672 0.6020 0.0000 0.0010 0.0004
MXNDArrayGetShape 4240 1.9980 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1792 15.7460 0.0070 0.0280 0.0088
MXNDArrayFree 20008 205.7570 0.0000 0.9140 0.0103
MXNet C API Calls 89632 576.9890 487.3580 576.9890 44.8155
MXNet C API Concurrency 179263 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1552 0.7280 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7760 4.0230 0.0000 0.0230 0.0005
MXNDArraySetGradState 7760 3.9590 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1552 0.6640 0.0000 0.0010 0.0004
MXAutogradBackwardEx 776 429.9010 0.5250 0.9450 0.5540
MXNDArrayGetContext 12424 5.4500 0.0000 0.0190 0.0004
MXImperativeInvokeEx 25976 667.1660 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3104 7591.7539 0.7010 3.7500 2.4458
FullyConnected 5376 518.7020 0.0080 0.3430 0.0965
argmax 1792 35.4960 0.0050 0.0480 0.0198
_backward_pick 1552 24.6650 0.0130 0.0320 0.0159
_backward_mul_scalar 1552 6.7940 0.0030 0.0220 0.0044
DeleteVariable 66400 474.3920 0.0010 0.9010 0.0071
Flatten 1792 75.1850 0.0080 0.1120 0.0420
Activation 7168 333.2690 0.0030 1.9440 0.0465
SetValueOp 1552 10.0720 0.0050 0.0180 0.0065
pick 1552 12.2300 0.0060 0.0280 0.0079
Pooling 3584 4999.5151 0.0640 3.1720 1.3950
Convolution 3584 20394.6465 0.2260 13.2050 5.6905
CopyCPU2CPU 1792 10.3380 0.0030 0.0360 0.0058
log_softmax 1552 48.3510 0.0150 0.0550 0.0312
_backward_copy 1552 80.2770 0.0220 0.0760 0.0517
_backward_Convolution 3104 28711.1289 2.6240 12.5610 9.2497
_mul_scalar 1552 5.5500 0.0020 0.0230 0.0036
_backward_log_softmax 1552 26.5010 0.0090 0.0400 0.0171
mean 4896 93.8410 0.0080 0.0730 0.0192
_backward_mean 1552 17.0740 0.0080 0.1030 0.0110
_backward_FullyConnected 4656 892.4260 0.0280 0.5940 0.1917
WaitForVar 3344 29.7170 0.0060 0.0280 0.0089
broadcast_equal 1792 8.9870 0.0030 0.0110 0.0050
sgd_update 15520 154.7030 0.0020 0.0790 0.0100
_backward_Activation 6208 459.0210 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61098 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5382 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 776 3102.5110 0.3060 73.5870 3.9981
MXNDArrayCreateEx 897 31.0430 0.0310 0.0640 0.0346
MXNDArrayWaitAll 777 16859.7090 5.6130 25.0610 21.6985
MXNDArraySyncCopyToCPU 1674 13243.0898 0.0600 24.1980 7.9110
MXNDArrayGetDType 1674 0.6020 0.0000 0.0010 0.0004
MXNDArrayGetShape 4245 2.0010 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1794 15.7640 0.0070 0.0280 0.0088
MXNDArrayFree 20031 205.9910 0.0000 0.9140 0.0103
MXNet C API Calls 89739 577.0960 487.3580 577.0960 44.8690
MXNet C API Concurrency 179477 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1554 0.7290 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7770 4.0270 0.0000 0.0230 0.0005
MXNDArraySetGradState 7770 3.9640 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1554 0.6640 0.0000 0.0010 0.0004
MXAutogradBackwardEx 777 430.4530 0.5250 0.9450 0.5540
MXNDArrayGetContext 12438 5.4540 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26007 667.9960 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3108 7601.5239 0.7010 3.7500 2.4458
FullyConnected 5382 519.2250 0.0080 0.3430 0.0965
argmax 1794 35.5330 0.0050 0.0480 0.0198
_backward_pick 1554 24.6950 0.0130 0.0320 0.0159
_backward_mul_scalar 1554 6.8020 0.0030 0.0220 0.0044
DeleteVariable 66480 474.9200 0.0010 0.9010 0.0071
Flatten 1794 75.2620 0.0080 0.1120 0.0420
Activation 7176 333.6370 0.0030 1.9440 0.0465
SetValueOp 1554 10.0840 0.0050 0.0180 0.0065
pick 1554 12.2460 0.0060 0.0280 0.0079
Pooling 3588 5004.9160 0.0640 3.1720 1.3949
Convolution 3588 20416.8359 0.2260 13.2050 5.6903
CopyCPU2CPU 1794 10.3470 0.0030 0.0360 0.0058
log_softmax 1554 48.4120 0.0150 0.0550 0.0312
_backward_copy 1554 80.3830 0.0220 0.0760 0.0517
_backward_Convolution 3108 28748.1289 2.6240 12.5610 9.2497
_mul_scalar 1554 5.5570 0.0020 0.0230 0.0036
_backward_log_softmax 1554 26.5320 0.0090 0.0400 0.0171
mean 4902 93.9600 0.0080 0.0730 0.0192
_backward_mean 1554 17.0960 0.0080 0.1030 0.0110
_backward_FullyConnected 4662 893.4990 0.0280 0.5940 0.1917
WaitForVar 3348 29.7510 0.0060 0.0280 0.0089
broadcast_equal 1794 8.9970 0.0030 0.0110 0.0050
sgd_update 15540 154.9010 0.0020 0.0790 0.0100
_backward_Activation 6216 459.6030 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61172 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5388 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 777 3106.4390 0.3060 73.5870 3.9980
MXNDArrayCreateEx 898 31.0790 0.0310 0.0640 0.0346
MXNDArrayWaitAll 778 16881.5898 5.6130 25.0610 21.6987
MXNDArraySyncCopyToCPU 1676 13257.1934 0.0600 24.1980 7.9100
MXNDArrayGetDType 1676 0.6030 0.0000 0.0010 0.0004
MXNDArrayGetShape 4250 2.0050 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1796 15.7820 0.0070 0.0280 0.0088
MXNDArrayFree 20054 206.2260 0.0000 0.9140 0.0103
MXNet C API Calls 89846 577.2030 487.3580 577.2030 44.9225
MXNet C API Concurrency 179691 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1556 0.7300 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7780 4.0330 0.0000 0.0230 0.0005
MXNDArraySetGradState 7780 3.9670 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1556 0.6650 0.0000 0.0010 0.0004
MXAutogradBackwardEx 778 431.0250 0.5250 0.9450 0.5540
MXNDArrayGetContext 12452 5.4600 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26038 668.8480 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3112 7611.3369 0.7010 3.7500 2.4458
FullyConnected 5388 519.7680 0.0080 0.3430 0.0965
argmax 1796 35.5710 0.0050 0.0480 0.0198
_backward_pick 1556 24.7250 0.0130 0.0320 0.0159
_backward_mul_scalar 1556 6.8100 0.0030 0.0220 0.0044
DeleteVariable 66560 475.4710 0.0010 0.9010 0.0071
Flatten 1796 75.3410 0.0080 0.1120 0.0419
Activation 7184 334.0060 0.0030 1.9440 0.0465
SetValueOp 1556 10.0970 0.0050 0.0180 0.0065
pick 1556 12.2610 0.0060 0.0280 0.0079
Pooling 3592 5010.3008 0.0640 3.1720 1.3948
Convolution 3592 20439.1113 0.2260 13.2050 5.6902
CopyCPU2CPU 1796 10.3570 0.0030 0.0360 0.0058
log_softmax 1556 48.4720 0.0150 0.0550 0.0312
_backward_copy 1556 80.4890 0.0220 0.0760 0.0517
_backward_Convolution 3112 28785.2363 2.6240 12.5610 9.2498
_mul_scalar 1556 5.5640 0.0020 0.0230 0.0036
_backward_log_softmax 1556 26.5640 0.0090 0.0400 0.0171
mean 4908 94.0770 0.0080 0.0730 0.0192
_backward_mean 1556 17.1180 0.0080 0.1030 0.0110
_backward_FullyConnected 4668 894.5690 0.0280 0.5940 0.1916
WaitForVar 3352 29.7880 0.0060 0.0280 0.0089
broadcast_equal 1796 9.0060 0.0030 0.0110 0.0050
sgd_update 15560 155.1000 0.0020 0.0790 0.0100
_backward_Activation 6224 460.1730 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61246 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5394 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 778 3110.4121 0.3060 73.5870 3.9980
MXNDArrayCreateEx 899 31.1140 0.0310 0.0640 0.0346
MXNDArrayWaitAll 779 16903.2891 5.6130 25.0610 21.6987
MXNDArraySyncCopyToCPU 1678 13271.3564 0.0600 24.1980 7.9090
MXNDArrayGetDType 1678 0.6050 0.0000 0.0010 0.0004
MXNDArrayGetShape 4255 2.0070 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1798 15.8010 0.0070 0.0280 0.0088
MXNDArrayFree 20077 206.4670 0.0000 0.9140 0.0103
MXNet C API Calls 89953 577.3100 487.3580 577.3100 44.9760
MXNet C API Concurrency 179905 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1558 0.7310 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7790 4.0410 0.0000 0.0230 0.0005
MXNDArraySetGradState 7790 3.9720 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1558 0.6670 0.0000 0.0010 0.0004
MXAutogradBackwardEx 779 431.5700 0.5250 0.9450 0.5540
MXNDArrayGetContext 12466 5.4670 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26069 669.6840 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3116 7620.9131 0.7010 3.7500 2.4457
FullyConnected 5394 520.3270 0.0080 0.3430 0.0965
argmax 1798 35.6080 0.0050 0.0480 0.0198
_backward_pick 1558 24.7560 0.0130 0.0320 0.0159
_backward_mul_scalar 1558 6.8190 0.0030 0.0220 0.0044
DeleteVariable 66640 476.0220 0.0010 0.9010 0.0071
Flatten 1798 75.4210 0.0080 0.1120 0.0419
Activation 7192 334.3770 0.0030 1.9440 0.0465
SetValueOp 1558 10.1110 0.0050 0.0180 0.0065
pick 1558 12.2760 0.0060 0.0280 0.0079
Pooling 3596 5015.6548 0.0640 3.1720 1.3948
Convolution 3596 20461.3281 0.2260 13.2050 5.6900
CopyCPU2CPU 1798 10.3670 0.0030 0.0360 0.0058
log_softmax 1558 48.5310 0.0150 0.0550 0.0311
_backward_copy 1558 80.5950 0.0220 0.0760 0.0517
_backward_Convolution 3116 28822.0488 2.6240 12.5610 9.2497
_mul_scalar 1558 5.5710 0.0020 0.0230 0.0036
_backward_log_softmax 1558 26.5960 0.0090 0.0400 0.0171
mean 4914 94.1930 0.0080 0.0730 0.0192
_backward_mean 1558 17.1400 0.0080 0.1030 0.0110
_backward_FullyConnected 4674 895.6390 0.0280 0.5940 0.1916
WaitForVar 3356 29.8240 0.0060 0.0280 0.0089
broadcast_equal 1798 9.0160 0.0030 0.0110 0.0050
sgd_update 15580 155.2500 0.0020 0.0790 0.0100
_backward_Activation 6232 460.8030 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61320 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5400 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 779 3114.3411 0.3060 73.5870 3.9979
MXNDArrayCreateEx 900 31.1500 0.0310 0.0640 0.0346
MXNDArrayWaitAll 780 16924.9453 5.6130 25.0610 21.6986
MXNDArraySyncCopyToCPU 1680 13285.9014 0.0600 24.1980 7.9083
MXNDArrayGetDType 1680 0.6050 0.0000 0.0010 0.0004
MXNDArrayGetShape 4260 2.0100 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1800 15.8190 0.0070 0.0280 0.0088
MXNDArrayFree 20100 206.7090 0.0000 0.9140 0.0103
MXNet C API Calls 90060 577.4170 487.3580 577.4170 45.0295
MXNet C API Concurrency 180119 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1560 0.7320 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7800 4.0460 0.0000 0.0230 0.0005
MXNDArraySetGradState 7800 3.9790 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1560 0.6680 0.0000 0.0010 0.0004
MXAutogradBackwardEx 780 432.1180 0.5250 0.9450 0.5540
MXNDArrayGetContext 12480 5.4720 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26100 670.5130 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3120 7630.5479 0.7010 3.7500 2.4457
FullyConnected 5400 521.0570 0.0080 0.3430 0.0965
argmax 1800 35.6520 0.0050 0.0480 0.0198
_backward_pick 1560 24.7900 0.0130 0.0320 0.0159
_backward_mul_scalar 1560 6.8270 0.0030 0.0220 0.0044
DeleteVariable 66720 476.5700 0.0010 0.9010 0.0071
Flatten 1800 75.4990 0.0080 0.1120 0.0419
Activation 7200 334.7520 0.0030 1.9440 0.0465
SetValueOp 1560 10.1240 0.0050 0.0180 0.0065
pick 1560 12.2940 0.0060 0.0280 0.0079
Pooling 3600 5021.0381 0.0640 3.1720 1.3947
Convolution 3600 20483.5195 0.2260 13.2050 5.6899
CopyCPU2CPU 1800 10.3770 0.0030 0.0360 0.0058
log_softmax 1560 48.6100 0.0150 0.0550 0.0312
_backward_copy 1560 80.6980 0.0220 0.0760 0.0517
_backward_Convolution 3120 28858.7578 2.6240 12.5610 9.2496
_mul_scalar 1560 5.5780 0.0020 0.0230 0.0036
_backward_log_softmax 1560 26.6390 0.0090 0.0400 0.0171
mean 4920 94.3000 0.0080 0.0730 0.0192
_backward_mean 1560 17.1620 0.0080 0.1030 0.0110
_backward_FullyConnected 4680 897.0500 0.0280 0.5940 0.1917
WaitForVar 3360 29.8600 0.0060 0.0280 0.0089
broadcast_equal 1800 9.0260 0.0030 0.0110 0.0050
sgd_update 15600 155.5220 0.0020 0.0790 0.0100
_backward_Activation 6240 461.3950 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61394 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5406 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 780 3118.2620 0.3060 73.5870 3.9978
MXNDArrayCreateEx 901 31.1860 0.0310 0.0640 0.0346
MXNDArrayWaitAll 781 16946.8340 5.6130 25.0610 21.6989
MXNDArraySyncCopyToCPU 1682 13300.1338 0.0600 24.1980 7.9073
MXNDArrayGetDType 1682 0.6060 0.0000 0.0010 0.0004
MXNDArrayGetShape 4265 2.0120 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1802 15.8370 0.0070 0.0280 0.0088
MXNDArrayFree 20123 206.9440 0.0000 0.9140 0.0103
MXNet C API Calls 90167 577.5240 487.3580 577.5240 45.0830
MXNet C API Concurrency 180333 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1562 0.7340 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7810 4.0520 0.0000 0.0230 0.0005
MXNDArraySetGradState 7810 3.9850 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1562 0.6690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 781 432.6650 0.5250 0.9450 0.5540
MXNDArrayGetContext 12494 5.4770 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26131 671.3310 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3124 7640.2798 0.7010 3.7500 2.4457
FullyConnected 5406 521.5790 0.0080 0.3430 0.0965
argmax 1802 35.6910 0.0050 0.0480 0.0198
_backward_pick 1562 24.8230 0.0130 0.0320 0.0159
_backward_mul_scalar 1562 6.8360 0.0030 0.0220 0.0044
DeleteVariable 66800 477.1080 0.0010 0.9010 0.0071
Flatten 1802 75.5780 0.0080 0.1120 0.0419
Activation 7208 335.1200 0.0030 1.9440 0.0465
SetValueOp 1562 10.1360 0.0050 0.0180 0.0065
pick 1562 12.3090 0.0060 0.0280 0.0079
Pooling 3604 5026.4009 0.0640 3.1720 1.3947
Convolution 3604 20505.6973 0.2260 13.2050 5.6897
CopyCPU2CPU 1802 10.3870 0.0030 0.0360 0.0058
log_softmax 1562 48.6700 0.0150 0.0550 0.0312
_backward_copy 1562 80.8030 0.0220 0.0760 0.0517
_backward_Convolution 3124 28895.8125 2.6240 12.5610 9.2496
_mul_scalar 1562 5.5850 0.0020 0.0230 0.0036
_backward_log_softmax 1562 26.6700 0.0090 0.0400 0.0171
mean 4926 94.4170 0.0080 0.0730 0.0192
_backward_mean 1562 17.1830 0.0080 0.1030 0.0110
_backward_FullyConnected 4686 898.1620 0.0280 0.5940 0.1917
WaitForVar 3364 29.8970 0.0060 0.0280 0.0089
broadcast_equal 1802 9.0370 0.0030 0.0110 0.0050
sgd_update 15620 155.6770 0.0020 0.0790 0.0100
_backward_Activation 6248 462.0320 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61468 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5412 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 781 3122.1941 0.3060 73.5870 3.9977
MXNDArrayCreateEx 902 31.2220 0.0310 0.0640 0.0346
MXNDArrayWaitAll 782 16968.7285 5.6130 25.0610 21.6991
MXNDArraySyncCopyToCPU 1684 13314.4609 0.0600 24.1980 7.9064
MXNDArrayGetDType 1684 0.6070 0.0000 0.0010 0.0004
MXNDArrayGetShape 4270 2.0130 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1804 15.8560 0.0070 0.0280 0.0088
MXNDArrayFree 20146 207.1780 0.0000 0.9140 0.0103
MXNet C API Calls 90274 577.6310 487.3580 577.6310 45.1365
MXNet C API Concurrency 180547 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1564 0.7350 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7820 4.0580 0.0000 0.0230 0.0005
MXNDArraySetGradState 7820 3.9890 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1564 0.6690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 782 433.2140 0.5250 0.9450 0.5540
MXNDArrayGetContext 12508 5.4840 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26162 672.1780 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3128 7650.0352 0.7010 3.7500 2.4457
FullyConnected 5412 522.1300 0.0080 0.3430 0.0965
argmax 1804 35.7290 0.0050 0.0480 0.0198
_backward_pick 1564 24.8590 0.0130 0.0320 0.0159
_backward_mul_scalar 1564 6.8440 0.0030 0.0220 0.0044
DeleteVariable 66880 477.6580 0.0010 0.9010 0.0071
Flatten 1804 75.6570 0.0080 0.1120 0.0419
Activation 7216 335.4890 0.0030 1.9440 0.0465
SetValueOp 1564 10.1490 0.0050 0.0180 0.0065
pick 1564 12.3250 0.0060 0.0280 0.0079
Pooling 3608 5031.8281 0.0640 3.1720 1.3946
Convolution 3608 20527.9395 0.2260 13.2050 5.6896
CopyCPU2CPU 1804 10.3970 0.0030 0.0360 0.0058
log_softmax 1564 48.7290 0.0150 0.0550 0.0312
_backward_copy 1564 80.9100 0.0220 0.0760 0.0517
_backward_Convolution 3128 28932.9434 2.6240 12.5610 9.2497
_mul_scalar 1564 5.5930 0.0020 0.0230 0.0036
_backward_log_softmax 1564 26.7020 0.0090 0.0400 0.0171
mean 4932 94.5360 0.0080 0.0730 0.0192
_backward_mean 1564 17.2040 0.0080 0.1030 0.0110
_backward_FullyConnected 4692 899.2300 0.0280 0.5940 0.1917
WaitForVar 3368 29.9320 0.0060 0.0280 0.0089
broadcast_equal 1804 9.0470 0.0030 0.0110 0.0050
sgd_update 15640 155.8350 0.0020 0.0790 0.0100
_backward_Activation 6256 462.6840 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61542 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5418 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 782 3126.1221 0.3060 73.5870 3.9976
MXNDArrayCreateEx 903 31.2570 0.0310 0.0640 0.0346
MXNDArrayWaitAll 783 16990.3965 5.6130 25.0610 21.6991
MXNDArraySyncCopyToCPU 1686 13328.7432 0.0600 24.1980 7.9055
MXNDArrayGetDType 1686 0.6080 0.0000 0.0010 0.0004
MXNDArrayGetShape 4275 2.0160 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1806 15.8740 0.0070 0.0280 0.0088
MXNDArrayFree 20169 207.4120 0.0000 0.9140 0.0103
MXNet C API Calls 90381 577.7380 487.3580 577.7380 45.1900
MXNet C API Concurrency 180761 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1566 0.7360 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7830 4.0620 0.0000 0.0230 0.0005
MXNDArraySetGradState 7830 3.9950 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1566 0.6690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 783 433.7600 0.5250 0.9450 0.5540
MXNDArrayGetContext 12522 5.4910 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26193 672.9910 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3132 7659.6538 0.7010 3.7500 2.4456
FullyConnected 5418 522.6500 0.0080 0.3430 0.0965
argmax 1806 35.7680 0.0050 0.0480 0.0198
_backward_pick 1566 24.8910 0.0130 0.0320 0.0159
_backward_mul_scalar 1566 6.8530 0.0030 0.0220 0.0044
DeleteVariable 66960 478.1960 0.0010 0.9010 0.0071
Flatten 1806 75.7360 0.0080 0.1120 0.0419
Activation 7224 335.9060 0.0030 1.9440 0.0465
SetValueOp 1566 10.1620 0.0050 0.0180 0.0065
pick 1566 12.3400 0.0060 0.0280 0.0079
Pooling 3612 5037.1821 0.0640 3.1720 1.3946
Convolution 3612 20550.1484 0.2260 13.2050 5.6894
CopyCPU2CPU 1806 10.4070 0.0030 0.0360 0.0058
log_softmax 1566 48.7890 0.0150 0.0550 0.0312
_backward_copy 1566 81.0150 0.0220 0.0760 0.0517
_backward_Convolution 3132 28969.6836 2.6240 12.5610 9.2496
_mul_scalar 1566 5.6010 0.0020 0.0230 0.0036
_backward_log_softmax 1566 26.7330 0.0090 0.0400 0.0171
mean 4938 94.6490 0.0080 0.0730 0.0192
_backward_mean 1566 17.2260 0.0080 0.1030 0.0110
_backward_FullyConnected 4698 900.3070 0.0280 0.5940 0.1916
WaitForVar 3372 29.9690 0.0060 0.0280 0.0089
broadcast_equal 1806 9.0570 0.0030 0.0110 0.0050
sgd_update 15660 156.0360 0.0020 0.0790 0.0100
_backward_Activation 6264 463.2800 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61616 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5424 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 783 3130.0410 0.3060 73.5870 3.9975
MXNDArrayCreateEx 904 31.2930 0.0310 0.0640 0.0346
MXNDArrayWaitAll 784 17012.1602 5.6130 25.0610 21.6992
MXNDArraySyncCopyToCPU 1688 13343.3184 0.0600 24.1980 7.9048
MXNDArrayGetDType 1688 0.6090 0.0000 0.0010 0.0004
MXNDArrayGetShape 4280 2.0170 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1808 15.8930 0.0070 0.0280 0.0088
MXNDArrayFree 20192 207.6430 0.0000 0.9140 0.0103
MXNet C API Calls 90488 577.8450 487.3580 577.8450 45.2435
MXNet C API Concurrency 180975 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1568 0.7370 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7840 4.0660 0.0000 0.0230 0.0005
MXNDArraySetGradState 7840 4.0000 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1568 0.6690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 784 434.3100 0.5250 0.9450 0.5540
MXNDArrayGetContext 12536 5.5000 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26224 673.8110 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3136 7669.4102 0.7010 3.7500 2.4456
FullyConnected 5424 523.4000 0.0080 0.3430 0.0965
argmax 1808 35.8130 0.0050 0.0480 0.0198
_backward_pick 1568 24.9240 0.0130 0.0320 0.0159
_backward_mul_scalar 1568 6.8620 0.0030 0.0220 0.0044
DeleteVariable 67040 478.7460 0.0010 0.9010 0.0071
Flatten 1808 75.8550 0.0080 0.1120 0.0420
Activation 7232 336.2890 0.0030 1.9440 0.0465
SetValueOp 1568 10.1750 0.0050 0.0180 0.0065
pick 1568 12.3570 0.0060 0.0280 0.0079
Pooling 3616 5042.5269 0.0640 3.1720 1.3945
Convolution 3616 20572.3574 0.2260 13.2050 5.6893
CopyCPU2CPU 1808 10.4170 0.0030 0.0360 0.0058
log_softmax 1568 48.8610 0.0150 0.0550 0.0312
_backward_copy 1568 81.1180 0.0220 0.0760 0.0517
_backward_Convolution 3136 29006.4551 2.6240 12.5610 9.2495
_mul_scalar 1568 5.6080 0.0020 0.0230 0.0036
_backward_log_softmax 1568 26.7750 0.0090 0.0400 0.0171
mean 4944 94.7580 0.0080 0.0730 0.0192
_backward_mean 1568 17.2480 0.0080 0.1030 0.0110
_backward_FullyConnected 4704 901.7120 0.0280 0.5940 0.1917
WaitForVar 3376 30.0050 0.0060 0.0280 0.0089
broadcast_equal 1808 9.0660 0.0030 0.0110 0.0050
sgd_update 15680 156.3050 0.0020 0.0790 0.0100
_backward_Activation 6272 463.8830 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61690 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5430 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 784 3133.9700 0.3060 73.5870 3.9974
MXNDArrayCreateEx 905 31.3290 0.0310 0.0640 0.0346
MXNDArrayWaitAll 785 17033.9805 5.6130 25.0610 21.6993
MXNDArraySyncCopyToCPU 1690 13357.5244 0.0600 24.1980 7.9039
MXNDArrayGetDType 1690 0.6100 0.0000 0.0010 0.0004
MXNDArrayGetShape 4285 2.0190 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1810 15.9120 0.0070 0.0280 0.0088
MXNDArrayFree 20215 207.8780 0.0000 0.9140 0.0103
MXNet C API Calls 90595 577.9520 487.3580 577.9520 45.2970
MXNet C API Concurrency 181189 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1570 0.7380 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7850 4.0700 0.0000 0.0230 0.0005
MXNDArraySetGradState 7850 4.0070 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1570 0.6700 0.0000 0.0010 0.0004
MXAutogradBackwardEx 785 434.8560 0.5250 0.9450 0.5540
MXNDArrayGetContext 12550 5.5070 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26255 674.6350 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3140 7679.0239 0.7010 3.7500 2.4455
FullyConnected 5430 523.9320 0.0080 0.3430 0.0965
argmax 1810 35.8510 0.0050 0.0480 0.0198
_backward_pick 1570 24.9550 0.0130 0.0320 0.0159
_backward_mul_scalar 1570 6.8710 0.0030 0.0220 0.0044
DeleteVariable 67120 479.2960 0.0010 0.9010 0.0071
Flatten 1810 75.9340 0.0080 0.1120 0.0420
Activation 7240 336.6600 0.0030 1.9440 0.0465
SetValueOp 1570 10.1890 0.0050 0.0180 0.0065
pick 1570 12.3720 0.0060 0.0280 0.0079
Pooling 3620 5047.9282 0.0640 3.1720 1.3945
Convolution 3620 20594.5156 0.2260 13.2050 5.6891
CopyCPU2CPU 1810 10.4270 0.0030 0.0360 0.0058
log_softmax 1570 48.9200 0.0150 0.0550 0.0312
_backward_copy 1570 81.2230 0.0220 0.0760 0.0517
_backward_Convolution 3140 29043.4922 2.6240 12.5610 9.2495
_mul_scalar 1570 5.6150 0.0020 0.0230 0.0036
_backward_log_softmax 1570 26.8070 0.0090 0.0400 0.0171
mean 4950 94.8750 0.0080 0.0730 0.0192
_backward_mean 1570 17.2700 0.0080 0.1030 0.0110
_backward_FullyConnected 4710 902.7810 0.0280 0.5940 0.1917
WaitForVar 3380 30.0420 0.0060 0.0280 0.0089
broadcast_equal 1810 9.0750 0.0030 0.0110 0.0050
sgd_update 15700 156.5110 0.0020 0.0790 0.0100
_backward_Activation 6280 464.4650 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61764 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5436 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 785 3137.8999 0.3060 73.5870 3.9973
MXNDArrayCreateEx 906 31.3650 0.0310 0.0640 0.0346
MXNDArrayWaitAll 786 17055.8711 5.6130 25.0610 21.6996
MXNDArraySyncCopyToCPU 1692 13371.7451 0.0600 24.1980 7.9029
MXNDArrayGetDType 1692 0.6100 0.0000 0.0010 0.0004
MXNDArrayGetShape 4290 2.0210 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1812 15.9300 0.0070 0.0280 0.0088
MXNDArrayFree 20238 208.1070 0.0000 0.9140 0.0103
MXNet C API Calls 90702 578.0590 487.3580 578.0590 45.3505
MXNet C API Concurrency 181403 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1572 0.7400 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7860 4.0780 0.0000 0.0230 0.0005
MXNDArraySetGradState 7860 4.0140 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1572 0.6720 0.0000 0.0010 0.0004
MXAutogradBackwardEx 786 435.4040 0.5250 0.9450 0.5539
MXNDArrayGetContext 12564 5.5130 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26286 675.4570 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3144 7688.6748 0.7010 3.7500 2.4455
FullyConnected 5436 524.4510 0.0080 0.3430 0.0965
argmax 1812 35.8880 0.0050 0.0480 0.0198
_backward_pick 1572 24.9860 0.0130 0.0320 0.0159
_backward_mul_scalar 1572 6.8790 0.0030 0.0220 0.0044
DeleteVariable 67200 479.8240 0.0010 0.9010 0.0071
Flatten 1812 76.0110 0.0080 0.1120 0.0419
Activation 7248 337.0280 0.0030 1.9440 0.0465
SetValueOp 1572 10.2020 0.0050 0.0180 0.0065
pick 1572 12.3860 0.0060 0.0280 0.0079
Pooling 3624 5053.2739 0.0640 3.1720 1.3944
Convolution 3624 20616.7812 0.2260 13.2050 5.6890
CopyCPU2CPU 1812 10.4360 0.0030 0.0360 0.0058
log_softmax 1572 48.9800 0.0150 0.0550 0.0312
_backward_copy 1572 81.3280 0.0220 0.0760 0.0517
_backward_Convolution 3144 29080.5938 2.6240 12.5610 9.2496
_mul_scalar 1572 5.6220 0.0020 0.0230 0.0036
_backward_log_softmax 1572 26.8400 0.0090 0.0400 0.0171
mean 4956 94.9950 0.0080 0.0730 0.0192
_backward_mean 1572 17.2910 0.0080 0.1030 0.0110
_backward_FullyConnected 4716 903.8530 0.0280 0.5940 0.1917
WaitForVar 3384 30.0770 0.0060 0.0280 0.0089
broadcast_equal 1812 9.0840 0.0030 0.0110 0.0050
sgd_update 15720 156.7160 0.0020 0.0790 0.0100
_backward_Activation 6288 465.0300 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61838 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5442 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 786 3141.8311 0.3060 73.5870 3.9972
MXNDArrayCreateEx 907 31.4000 0.0310 0.0640 0.0346
MXNDArrayWaitAll 787 17077.5352 5.6130 25.0610 21.6995
MXNDArraySyncCopyToCPU 1694 13385.9854 0.0600 24.1980 7.9020
MXNDArrayGetDType 1694 0.6100 0.0000 0.0010 0.0004
MXNDArrayGetShape 4295 2.0230 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1814 15.9480 0.0070 0.0280 0.0088
MXNDArrayFree 20261 208.3430 0.0000 0.9140 0.0103
MXNet C API Calls 90809 578.1660 487.3580 578.1660 45.4040
MXNet C API Concurrency 181617 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1574 0.7410 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7870 4.0840 0.0000 0.0230 0.0005
MXNDArraySetGradState 7870 4.0180 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1574 0.6730 0.0000 0.0010 0.0004
MXAutogradBackwardEx 787 435.9520 0.5250 0.9450 0.5539
MXNDArrayGetContext 12578 5.5200 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26317 676.2830 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3148 7698.3408 0.7010 3.7500 2.4455
FullyConnected 5442 524.9710 0.0080 0.3430 0.0965
argmax 1814 35.9260 0.0050 0.0480 0.0198
_backward_pick 1574 25.0170 0.0130 0.0320 0.0159
_backward_mul_scalar 1574 6.8880 0.0030 0.0220 0.0044
DeleteVariable 67280 480.3690 0.0010 0.9010 0.0071
Flatten 1814 76.0890 0.0080 0.1120 0.0419
Activation 7256 337.3980 0.0030 1.9440 0.0465
SetValueOp 1574 10.2150 0.0050 0.0180 0.0065
pick 1574 12.4010 0.0060 0.0280 0.0079
Pooling 3628 5058.6641 0.0640 3.1720 1.3943
Convolution 3628 20638.9492 0.2260 13.2050 5.6888
CopyCPU2CPU 1814 10.4450 0.0030 0.0360 0.0058
log_softmax 1574 49.0390 0.0150 0.0550 0.0312
_backward_copy 1574 81.4330 0.0220 0.0760 0.0517
_backward_Convolution 3148 29117.3223 2.6240 12.5610 9.2495
_mul_scalar 1574 5.6290 0.0020 0.0230 0.0036
_backward_log_softmax 1574 26.8720 0.0090 0.0400 0.0171
mean 4962 95.1130 0.0080 0.0730 0.0192
_backward_mean 1574 17.3130 0.0080 0.1030 0.0110
_backward_FullyConnected 4722 904.9230 0.0280 0.5940 0.1916
WaitForVar 3388 30.1130 0.0060 0.0280 0.0089
broadcast_equal 1814 9.0940 0.0030 0.0110 0.0050
sgd_update 15740 156.9220 0.0020 0.0790 0.0100
_backward_Activation 6296 465.5980 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61912 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5448 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 787 3145.7539 0.3060 73.5870 3.9971
MXNDArrayCreateEx 908 31.4360 0.0310 0.0640 0.0346
MXNDArrayWaitAll 788 17099.2793 5.6130 25.0610 21.6996
MXNDArraySyncCopyToCPU 1696 13400.5410 0.0600 24.1980 7.9013
MXNDArrayGetDType 1696 0.6110 0.0000 0.0010 0.0004
MXNDArrayGetShape 4300 2.0270 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1816 15.9670 0.0070 0.0280 0.0088
MXNDArrayFree 20284 208.5780 0.0000 0.9140 0.0103
MXNet C API Calls 90916 578.2730 487.3580 578.2730 45.4575
MXNet C API Concurrency 181831 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1576 0.7410 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7880 4.0890 0.0000 0.0230 0.0005
MXNDArraySetGradState 7880 4.0230 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1576 0.6730 0.0000 0.0010 0.0004
MXAutogradBackwardEx 788 436.5010 0.5250 0.9450 0.5539
MXNDArrayGetContext 12592 5.5270 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26348 677.1020 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3152 7708.0249 0.7010 3.7500 2.4454
FullyConnected 5448 525.6980 0.0080 0.3430 0.0965
argmax 1816 35.9710 0.0050 0.0480 0.0198
_backward_pick 1576 25.0490 0.0130 0.0320 0.0159
_backward_mul_scalar 1576 6.8970 0.0030 0.0220 0.0044
DeleteVariable 67360 480.9040 0.0010 0.9010 0.0071
Flatten 1816 76.1680 0.0080 0.1120 0.0419
Activation 7264 337.8210 0.0030 1.9440 0.0465
SetValueOp 1576 10.2270 0.0050 0.0180 0.0065
pick 1576 12.4170 0.0060 0.0280 0.0079
Pooling 3632 5064.0200 0.0640 3.1720 1.3943
Convolution 3632 20661.1270 0.2260 13.2050 5.6886
CopyCPU2CPU 1816 10.4540 0.0030 0.0360 0.0058
log_softmax 1576 49.1110 0.0150 0.0550 0.0312
_backward_copy 1576 81.5350 0.0220 0.0760 0.0517
_backward_Convolution 3152 29154.1250 2.6240 12.5610 9.2494
_mul_scalar 1576 5.6370 0.0020 0.0230 0.0036
_backward_log_softmax 1576 26.9140 0.0090 0.0400 0.0171
mean 4968 95.2220 0.0080 0.0730 0.0192
_backward_mean 1576 17.3360 0.0080 0.1030 0.0110
_backward_FullyConnected 4728 906.3580 0.0280 0.5940 0.1917
WaitForVar 3392 30.1480 0.0060 0.0280 0.0089
broadcast_equal 1816 9.1040 0.0030 0.0110 0.0050
sgd_update 15760 157.1950 0.0020 0.0790 0.0100
_backward_Activation 6304 466.1930 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 61986 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5454 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 788 3149.6541 0.3060 73.5870 3.9970
MXNDArrayCreateEx 909 31.4710 0.0310 0.0640 0.0346
MXNDArrayWaitAll 789 17121.1152 5.6130 25.0610 21.6998
MXNDArraySyncCopyToCPU 1698 13414.6924 0.0600 24.1980 7.9003
MXNDArrayGetDType 1698 0.6130 0.0000 0.0010 0.0004
MXNDArrayGetShape 4305 2.0280 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1818 15.9840 0.0070 0.0280 0.0088
MXNDArrayFree 20307 208.8190 0.0000 0.9140 0.0103
MXNet C API Calls 91023 578.3800 487.3580 578.3800 45.5110
MXNet C API Concurrency 182045 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1578 0.7430 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7890 4.0950 0.0000 0.0230 0.0005
MXNDArraySetGradState 7890 4.0300 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1578 0.6730 0.0000 0.0010 0.0004
MXAutogradBackwardEx 789 437.0830 0.5250 0.9450 0.5540
MXNDArrayGetContext 12606 5.5300 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26379 677.9230 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3156 7717.5659 0.7010 3.7500 2.4454
FullyConnected 5454 526.2380 0.0080 0.3430 0.0965
argmax 1818 36.0080 0.0050 0.0480 0.0198
_backward_pick 1578 25.0820 0.0130 0.0320 0.0159
_backward_mul_scalar 1578 6.9050 0.0030 0.0220 0.0044
DeleteVariable 67440 481.4550 0.0010 0.9010 0.0071
Flatten 1818 76.2470 0.0080 0.1120 0.0419
Activation 7272 338.1940 0.0030 1.9440 0.0465
SetValueOp 1578 10.2410 0.0050 0.0180 0.0065
pick 1578 12.4320 0.0060 0.0280 0.0079
Pooling 3636 5069.4521 0.0640 3.1720 1.3942
Convolution 3636 20683.2637 0.2260 13.2050 5.6885
CopyCPU2CPU 1818 10.4640 0.0030 0.0360 0.0058
log_softmax 1578 49.1710 0.0150 0.0550 0.0312
_backward_copy 1578 81.6410 0.0220 0.0760 0.0517
_backward_Convolution 3156 29191.1934 2.6240 12.5610 9.2494
_mul_scalar 1578 5.6450 0.0020 0.0230 0.0036
_backward_log_softmax 1578 26.9460 0.0090 0.0400 0.0171
mean 4974 95.2950 0.0080 0.0730 0.0192
_backward_mean 1578 17.3570 0.0080 0.1030 0.0110
_backward_FullyConnected 4734 907.4340 0.0280 0.5940 0.1917
WaitForVar 3396 30.1870 0.0060 0.0280 0.0089
broadcast_equal 1818 9.1140 0.0030 0.0110 0.0050
sgd_update 15780 157.3990 0.0020 0.0790 0.0100
_backward_Activation 6312 466.7630 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62060 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5460 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 789 3153.5779 0.3060 73.5870 3.9969
MXNDArrayCreateEx 910 31.5070 0.0310 0.0640 0.0346
MXNDArrayWaitAll 790 17142.9336 5.6130 25.0610 21.6999
MXNDArraySyncCopyToCPU 1700 13428.9678 0.0600 24.1980 7.8994
MXNDArrayGetDType 1700 0.6130 0.0000 0.0010 0.0004
MXNDArrayGetShape 4310 2.0310 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1820 16.0010 0.0070 0.0280 0.0088
MXNDArrayFree 20330 209.0590 0.0000 0.9140 0.0103
MXNet C API Calls 91130 578.4870 487.3580 578.4870 45.5645
MXNet C API Concurrency 182259 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1580 0.7450 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7900 4.1000 0.0000 0.0230 0.0005
MXNDArraySetGradState 7900 4.0340 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1580 0.6750 0.0000 0.0010 0.0004
MXAutogradBackwardEx 790 437.6280 0.5250 0.9450 0.5540
MXNDArrayGetContext 12620 5.5390 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26410 678.7650 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3160 7727.2109 0.7010 3.7500 2.4453
FullyConnected 5460 526.7580 0.0080 0.3430 0.0965
argmax 1820 36.0460 0.0050 0.0480 0.0198
_backward_pick 1580 25.1130 0.0130 0.0320 0.0159
_backward_mul_scalar 1580 6.9140 0.0030 0.0220 0.0044
DeleteVariable 67520 481.9880 0.0010 0.9010 0.0071
Flatten 1820 76.3270 0.0080 0.1120 0.0419
Activation 7280 338.5630 0.0030 1.9440 0.0465
SetValueOp 1580 10.2540 0.0050 0.0180 0.0065
pick 1580 12.4480 0.0060 0.0280 0.0079
Pooling 3640 5074.8442 0.0640 3.1720 1.3942
Convolution 3640 20705.4863 0.2260 13.2050 5.6883
CopyCPU2CPU 1820 10.4740 0.0030 0.0360 0.0058
log_softmax 1580 49.2320 0.0150 0.0550 0.0312
_backward_copy 1580 81.7470 0.0220 0.0760 0.0517
_backward_Convolution 3160 29228.2598 2.6240 12.5610 9.2494
_mul_scalar 1580 5.6520 0.0020 0.0230 0.0036
_backward_log_softmax 1580 26.9780 0.0090 0.0400 0.0171
mean 4980 95.4140 0.0080 0.0730 0.0192
_backward_mean 1580 17.3780 0.0080 0.1030 0.0110
_backward_FullyConnected 4740 908.5060 0.0280 0.5940 0.1917
WaitForVar 3400 30.2220 0.0060 0.0280 0.0089
broadcast_equal 1820 9.1240 0.0030 0.0110 0.0050
sgd_update 15800 157.6020 0.0020 0.0790 0.0100
_backward_Activation 6320 467.3440 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62134 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5466 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 790 3157.5071 0.3060 73.5870 3.9968
MXNDArrayCreateEx 911 31.5420 0.0310 0.0640 0.0346
MXNDArrayWaitAll 791 17164.6758 5.6130 25.0610 21.7000
MXNDArraySyncCopyToCPU 1702 13443.1934 0.0600 24.1980 7.8985
MXNDArrayGetDType 1702 0.6150 0.0000 0.0010 0.0004
MXNDArrayGetShape 4315 2.0330 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1822 16.0200 0.0070 0.0280 0.0088
MXNDArrayFree 20353 209.2950 0.0000 0.9140 0.0103
MXNet C API Calls 91237 578.5940 487.3580 578.5940 45.6180
MXNet C API Concurrency 182473 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1582 0.7460 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7910 4.1070 0.0000 0.0230 0.0005
MXNDArraySetGradState 7910 4.0410 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1582 0.6760 0.0000 0.0010 0.0004
MXAutogradBackwardEx 791 438.1770 0.5250 0.9450 0.5540
MXNDArrayGetContext 12634 5.5470 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26441 679.5930 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3164 7736.9082 0.7010 3.7500 2.4453
FullyConnected 5466 527.3390 0.0080 0.3430 0.0965
argmax 1822 36.0840 0.0050 0.0480 0.0198
_backward_pick 1582 25.1430 0.0130 0.0320 0.0159
_backward_mul_scalar 1582 6.9220 0.0030 0.0220 0.0044
DeleteVariable 67600 482.5370 0.0010 0.9010 0.0071
Flatten 1822 76.4070 0.0080 0.1120 0.0419
Activation 7288 338.9340 0.0030 1.9440 0.0465
SetValueOp 1582 10.2670 0.0050 0.0180 0.0065
pick 1582 12.4630 0.0060 0.0280 0.0079
Pooling 3644 5080.1631 0.0640 3.1720 1.3941
Convolution 3644 20727.6914 0.2260 13.2050 5.6882
CopyCPU2CPU 1822 10.4850 0.0030 0.0360 0.0058
log_softmax 1582 49.2920 0.0150 0.0550 0.0312
_backward_copy 1582 81.8540 0.0220 0.0760 0.0517
_backward_Convolution 3164 29265.0801 2.6240 12.5610 9.2494
_mul_scalar 1582 5.6590 0.0020 0.0230 0.0036
_backward_log_softmax 1582 27.0110 0.0090 0.0400 0.0171
mean 4986 95.5310 0.0080 0.0730 0.0192
_backward_mean 1582 17.3990 0.0080 0.1030 0.0110
_backward_FullyConnected 4746 909.5810 0.0280 0.5940 0.1917
WaitForVar 3404 30.2580 0.0060 0.0280 0.0089
broadcast_equal 1822 9.1340 0.0030 0.0110 0.0050
sgd_update 15820 157.8050 0.0020 0.0790 0.0100
_backward_Activation 6328 467.9140 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62208 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5472 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 791 3161.4431 0.3060 73.5870 3.9968
MXNDArrayCreateEx 912 31.5780 0.0310 0.0640 0.0346
MXNDArrayWaitAll 792 17186.3730 5.6130 25.0610 21.7000
MXNDArraySyncCopyToCPU 1704 13457.6934 0.0600 24.1980 7.8977
MXNDArrayGetDType 1704 0.6160 0.0000 0.0010 0.0004
MXNDArrayGetShape 4320 2.0350 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1824 16.0380 0.0070 0.0280 0.0088
MXNDArrayFree 20376 209.5430 0.0000 0.9140 0.0103
MXNet C API Calls 91344 578.7010 487.3580 578.7010 45.6715
MXNet C API Concurrency 182687 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1584 0.7480 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7920 4.1110 0.0000 0.0230 0.0005
MXNDArraySetGradState 7920 4.0460 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1584 0.6780 0.0000 0.0010 0.0004
MXAutogradBackwardEx 792 438.7260 0.5250 0.9450 0.5539
MXNDArrayGetContext 12648 5.5530 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26472 680.4230 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3168 7746.6382 0.7010 3.7500 2.4453
FullyConnected 5472 528.0670 0.0080 0.3430 0.0965
argmax 1824 36.1290 0.0050 0.0480 0.0198
_backward_pick 1584 25.1750 0.0130 0.0320 0.0159
_backward_mul_scalar 1584 6.9310 0.0030 0.0220 0.0044
DeleteVariable 67680 483.0940 0.0010 0.9010 0.0071
Flatten 1824 76.4850 0.0080 0.1120 0.0419
Activation 7296 339.3170 0.0030 1.9440 0.0465
SetValueOp 1584 10.2800 0.0050 0.0180 0.0065
pick 1584 12.4790 0.0060 0.0280 0.0079
Pooling 3648 5085.5161 0.0640 3.1720 1.3941
Convolution 3648 20749.8652 0.2260 13.2050 5.6880
CopyCPU2CPU 1824 10.4950 0.0030 0.0360 0.0058
log_softmax 1584 49.3630 0.0150 0.0550 0.0312
_backward_copy 1584 81.9570 0.0220 0.0760 0.0517
_backward_Convolution 3168 29301.8027 2.6240 12.5610 9.2493
_mul_scalar 1584 5.6660 0.0020 0.0230 0.0036
_backward_log_softmax 1584 27.0540 0.0090 0.0400 0.0171
mean 4992 95.6390 0.0080 0.0730 0.0192
_backward_mean 1584 17.4210 0.0080 0.1030 0.0110
_backward_FullyConnected 4752 910.9830 0.0280 0.5940 0.1917
WaitForVar 3408 30.2940 0.0060 0.0280 0.0089
broadcast_equal 1824 9.1430 0.0030 0.0110 0.0050
sgd_update 15840 158.0730 0.0020 0.0790 0.0100
_backward_Activation 6336 468.5290 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62282 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5478 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 792 3165.3660 0.3060 73.5870 3.9967
MXNDArrayCreateEx 913 31.6140 0.0310 0.0640 0.0346
MXNDArrayWaitAll 793 17208.1270 5.6130 25.0610 21.7000
MXNDArraySyncCopyToCPU 1706 13471.7422 0.0600 24.1980 7.8967
MXNDArrayGetDType 1706 0.6170 0.0000 0.0010 0.0004
MXNDArrayGetShape 4325 2.0360 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1826 16.0570 0.0070 0.0280 0.0088
MXNDArrayFree 20399 209.7840 0.0000 0.9140 0.0103
MXNet C API Calls 91451 578.8080 487.3580 578.8080 45.7250
MXNet C API Concurrency 182901 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1586 0.7480 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7930 4.1180 0.0000 0.0230 0.0005
MXNDArraySetGradState 7930 4.0500 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1586 0.6790 0.0000 0.0010 0.0004
MXAutogradBackwardEx 793 439.2950 0.5250 0.9450 0.5540
MXNDArrayGetContext 12662 5.5610 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26503 681.2700 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3172 7756.3599 0.7010 3.7500 2.4453
FullyConnected 5478 528.6120 0.0080 0.3430 0.0965
argmax 1826 36.1670 0.0050 0.0480 0.0198
_backward_pick 1586 25.2070 0.0130 0.0320 0.0159
_backward_mul_scalar 1586 6.9400 0.0030 0.0220 0.0044
DeleteVariable 67760 483.6280 0.0010 0.9010 0.0071
Flatten 1826 76.5660 0.0080 0.1120 0.0419
Activation 7304 339.6880 0.0030 1.9440 0.0465
SetValueOp 1586 10.2950 0.0050 0.0180 0.0065
pick 1586 12.4950 0.0060 0.0280 0.0079
Pooling 3652 5090.8638 0.0640 3.1720 1.3940
Convolution 3652 20772.0566 0.2260 13.2050 5.6879
CopyCPU2CPU 1826 10.5050 0.0030 0.0360 0.0058
log_softmax 1586 49.4220 0.0150 0.0550 0.0312
_backward_copy 1586 82.0630 0.0220 0.0760 0.0517
_backward_Convolution 3172 29338.8672 2.6240 12.5610 9.2493
_mul_scalar 1586 5.6720 0.0020 0.0230 0.0036
_backward_log_softmax 1586 27.0850 0.0090 0.0400 0.0171
mean 4998 95.7590 0.0080 0.0730 0.0192
_backward_mean 1586 17.4420 0.0080 0.1030 0.0110
_backward_FullyConnected 4758 912.0940 0.0280 0.5940 0.1917
WaitForVar 3412 30.3300 0.0060 0.0280 0.0089
broadcast_equal 1826 9.1520 0.0030 0.0110 0.0050
sgd_update 15860 158.2760 0.0020 0.0790 0.0100
_backward_Activation 6344 469.0990 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62356 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5484 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 793 3169.3049 0.3060 73.5870 3.9966
MXNDArrayCreateEx 914 31.6500 0.0310 0.0640 0.0346
MXNDArrayWaitAll 794 17230.0664 5.6130 25.0610 21.7003
MXNDArraySyncCopyToCPU 1708 13486.0459 0.0600 24.1980 7.8958
MXNDArrayGetDType 1708 0.6170 0.0000 0.0010 0.0004
MXNDArrayGetShape 4330 2.0390 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1828 16.0760 0.0070 0.0280 0.0088
MXNDArrayFree 20422 210.0270 0.0000 0.9140 0.0103
MXNet C API Calls 91558 578.9150 487.3580 578.9150 45.7785
MXNet C API Concurrency 183115 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1588 0.7490 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7940 4.1210 0.0000 0.0230 0.0005
MXNDArraySetGradState 7940 4.0540 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1588 0.6790 0.0000 0.0010 0.0004
MXAutogradBackwardEx 794 439.8450 0.5250 0.9450 0.5540
MXNDArrayGetContext 12676 5.5680 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26534 682.1040 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3176 7766.2588 0.7010 3.7500 2.4453
FullyConnected 5484 529.1380 0.0080 0.3430 0.0965
argmax 1828 36.2060 0.0050 0.0480 0.0198
_backward_pick 1588 25.2370 0.0130 0.0320 0.0159
_backward_mul_scalar 1588 6.9490 0.0030 0.0220 0.0044
DeleteVariable 67840 484.1750 0.0010 0.9010 0.0071
Flatten 1828 76.6450 0.0080 0.1120 0.0419
Activation 7312 340.0760 0.0030 1.9440 0.0465
SetValueOp 1588 10.3080 0.0050 0.0180 0.0065
pick 1588 12.5110 0.0060 0.0280 0.0079
Pooling 3656 5096.2529 0.0640 3.1720 1.3939
Convolution 3656 20794.3262 0.2260 13.2050 5.6877
CopyCPU2CPU 1828 10.5150 0.0030 0.0360 0.0058
log_softmax 1588 49.4820 0.0150 0.0550 0.0312
_backward_copy 1588 82.1690 0.0220 0.0760 0.0517
_backward_Convolution 3176 29375.9336 2.6240 12.5610 9.2493
_mul_scalar 1588 5.6780 0.0020 0.0230 0.0036
_backward_log_softmax 1588 27.1170 0.0090 0.0400 0.0171
mean 5004 95.8750 0.0080 0.0730 0.0192
_backward_mean 1588 17.4630 0.0080 0.1030 0.0110
_backward_FullyConnected 4764 913.1730 0.0280 0.5940 0.1917
WaitForVar 3416 30.3660 0.0060 0.0280 0.0089
broadcast_equal 1828 9.1610 0.0030 0.0110 0.0050
sgd_update 15880 158.4340 0.0020 0.0790 0.0100
_backward_Activation 6352 469.7480 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62430 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5490 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 794 3170.5850 0.3060 73.5870 3.9932
MXNDArrayCreateEx 915 31.6860 0.0310 0.0640 0.0346
MXNDArrayWaitAll 795 17251.8223 5.6130 25.0610 21.7004
MXNDArraySyncCopyToCPU 1710 13500.2588 0.0600 24.1980 7.8949
MXNDArrayGetDType 1710 0.6170 0.0000 0.0010 0.0004
MXNDArrayGetShape 4335 2.0420 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1830 16.0950 0.0070 0.0280 0.0088
MXNDArrayFree 20445 210.2630 0.0000 0.9140 0.0103
MXNet C API Calls 91665 579.0220 487.3580 579.0220 45.8320
MXNet C API Concurrency 183329 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1590 0.7490 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7950 4.1270 0.0000 0.0230 0.0005
MXNDArraySetGradState 7950 4.0580 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1590 0.6800 0.0000 0.0010 0.0004
MXAutogradBackwardEx 795 440.4260 0.5250 0.9450 0.5540
MXNDArrayGetContext 12690 5.5750 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26565 682.9280 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3180 7775.9629 0.7010 3.7500 2.4453
FullyConnected 5490 529.6730 0.0080 0.3430 0.0965
argmax 1830 36.2450 0.0050 0.0480 0.0198
_backward_pick 1590 25.2680 0.0130 0.0320 0.0159
_backward_mul_scalar 1590 6.9570 0.0030 0.0220 0.0044
DeleteVariable 67920 484.7190 0.0010 0.9010 0.0071
Flatten 1830 76.7230 0.0080 0.1120 0.0419
Activation 7320 340.4470 0.0030 1.9440 0.0465
SetValueOp 1590 10.3210 0.0050 0.0180 0.0065
pick 1590 12.5260 0.0060 0.0280 0.0079
Pooling 3660 5101.6338 0.0640 3.1720 1.3939
Convolution 3660 20816.5312 0.2260 13.2050 5.6876
CopyCPU2CPU 1830 10.5250 0.0030 0.0360 0.0058
log_softmax 1590 49.5420 0.0150 0.0550 0.0312
_backward_copy 1590 82.2740 0.0220 0.0760 0.0517
_backward_Convolution 3180 29412.7754 2.6240 12.5610 9.2493
_mul_scalar 1590 5.6850 0.0020 0.0230 0.0036
_backward_log_softmax 1590 27.1500 0.0090 0.0400 0.0171
mean 5010 95.9930 0.0080 0.0730 0.0192
_backward_mean 1590 17.4850 0.0080 0.1030 0.0110
_backward_FullyConnected 4770 914.2420 0.0280 0.5940 0.1917
WaitForVar 3420 30.4000 0.0060 0.0280 0.0089
broadcast_equal 1830 9.1700 0.0030 0.0110 0.0050
sgd_update 15900 158.6390 0.0020 0.0790 0.0100
_backward_Activation 6360 470.3290 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62504 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5496 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 795 3174.5090 0.3060 73.5870 3.9931
MXNDArrayCreateEx 916 31.7220 0.0310 0.0640 0.0346
MXNDArrayWaitAll 796 17273.5117 5.6130 25.0610 21.7004
MXNDArraySyncCopyToCPU 1712 13514.8594 0.0600 24.1980 7.8942
MXNDArrayGetDType 1712 0.6170 0.0000 0.0010 0.0004
MXNDArrayGetShape 4340 2.0420 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1832 16.1130 0.0070 0.0280 0.0088
MXNDArrayFree 20468 210.5020 0.0000 0.9140 0.0103
MXNet C API Calls 91772 579.1290 487.3580 579.1290 45.8855
MXNet C API Concurrency 183543 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1592 0.7510 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7960 4.1330 0.0000 0.0230 0.0005
MXNDArraySetGradState 7960 4.0630 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1592 0.6800 0.0000 0.0010 0.0004
MXAutogradBackwardEx 796 440.9740 0.5250 0.9450 0.5540
MXNDArrayGetContext 12704 5.5820 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26596 683.7530 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3184 7785.7339 0.7010 3.7500 2.4453
FullyConnected 5496 530.3990 0.0080 0.3430 0.0965
argmax 1832 36.2890 0.0050 0.0480 0.0198
_backward_pick 1592 25.3000 0.0130 0.0320 0.0159
_backward_mul_scalar 1592 6.9660 0.0030 0.0220 0.0044
DeleteVariable 68000 485.2660 0.0010 0.9010 0.0071
Flatten 1832 76.8020 0.0080 0.1120 0.0419
Activation 7328 340.8280 0.0030 1.9440 0.0465
SetValueOp 1592 10.3340 0.0050 0.0180 0.0065
pick 1592 12.5410 0.0060 0.0280 0.0079
Pooling 3664 5107.0210 0.0640 3.1720 1.3938
Convolution 3664 20838.7305 0.2260 13.2050 5.6874
CopyCPU2CPU 1832 10.5340 0.0030 0.0360 0.0058
log_softmax 1592 49.6140 0.0150 0.0550 0.0312
_backward_copy 1592 82.3770 0.0220 0.0760 0.0517
_backward_Convolution 3184 29449.4727 2.6240 12.5610 9.2492
_mul_scalar 1592 5.6930 0.0020 0.0230 0.0036
_backward_log_softmax 1592 27.1900 0.0090 0.0400 0.0171
mean 5016 96.0940 0.0080 0.0730 0.0192
_backward_mean 1592 17.5070 0.0080 0.1030 0.0110
_backward_FullyConnected 4776 915.6480 0.0280 0.5940 0.1917
WaitForVar 3424 30.4360 0.0060 0.0280 0.0089
broadcast_equal 1832 9.1800 0.0030 0.0110 0.0050
sgd_update 15920 158.9100 0.0020 0.0790 0.0100
_backward_Activation 6368 470.9300 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62578 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5502 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 796 3178.4360 0.3060 73.5870 3.9930
MXNDArrayCreateEx 917 31.7580 0.0310 0.0640 0.0346
MXNDArrayWaitAll 797 17295.4434 5.6130 25.0610 21.7007
MXNDArraySyncCopyToCPU 1714 13529.0537 0.0600 24.1980 7.8933
MXNDArrayGetDType 1714 0.6170 0.0000 0.0010 0.0004
MXNDArrayGetShape 4345 2.0440 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1834 16.1320 0.0070 0.0280 0.0088
MXNDArrayFree 20491 210.7320 0.0000 0.9140 0.0103
MXNet C API Calls 91879 579.2360 487.3580 579.2360 45.9390
MXNet C API Concurrency 183757 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1594 0.7520 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7970 4.1400 0.0000 0.0230 0.0005
MXNDArraySetGradState 7970 4.0670 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1594 0.6810 0.0000 0.0010 0.0004
MXAutogradBackwardEx 797 441.5230 0.5250 0.9450 0.5540
MXNDArrayGetContext 12718 5.5880 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26627 684.5800 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3188 7795.4580 0.7010 3.7500 2.4453
FullyConnected 5502 530.9480 0.0080 0.3430 0.0965
argmax 1834 36.3290 0.0050 0.0480 0.0198
_backward_pick 1594 25.3310 0.0130 0.0320 0.0159
_backward_mul_scalar 1594 6.9750 0.0030 0.0220 0.0044
DeleteVariable 68080 485.8400 0.0010 0.9010 0.0071
Flatten 1834 76.8810 0.0080 0.1120 0.0419
Activation 7336 341.1970 0.0030 1.9440 0.0465
SetValueOp 1594 10.3470 0.0050 0.0180 0.0065
pick 1594 12.5560 0.0060 0.0280 0.0079
Pooling 3668 5112.3291 0.0640 3.1720 1.3938
Convolution 3668 20860.9219 0.2260 13.2050 5.6873
CopyCPU2CPU 1834 10.5440 0.0030 0.0360 0.0057
log_softmax 1594 49.6740 0.0150 0.0550 0.0312
_backward_copy 1594 82.4830 0.0220 0.0760 0.0517
_backward_Convolution 3188 29486.5938 2.6240 12.5610 9.2492
_mul_scalar 1594 5.7010 0.0020 0.0230 0.0036
_backward_log_softmax 1594 27.2220 0.0090 0.0400 0.0171
mean 5022 96.2110 0.0080 0.0730 0.0192
_backward_mean 1594 17.5280 0.0080 0.1030 0.0110
_backward_FullyConnected 4782 916.7220 0.0280 0.5940 0.1917
WaitForVar 3428 30.4710 0.0060 0.0280 0.0089
broadcast_equal 1834 9.1890 0.0030 0.0110 0.0050
sgd_update 15940 159.1180 0.0020 0.0790 0.0100
_backward_Activation 6376 471.5090 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62652 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5508 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 797 3182.3601 0.3060 73.5870 3.9929
MXNDArrayCreateEx 918 31.7940 0.0310 0.0640 0.0346
MXNDArrayWaitAll 798 17317.2754 5.6130 25.0610 21.7008
MXNDArraySyncCopyToCPU 1716 13543.3359 0.0600 24.1980 7.8924
MXNDArrayGetDType 1716 0.6180 0.0000 0.0010 0.0004
MXNDArrayGetShape 4350 2.0470 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1836 16.1500 0.0070 0.0280 0.0088
MXNDArrayFree 20514 210.9660 0.0000 0.9140 0.0103
MXNet C API Calls 91986 579.3430 487.3580 579.3430 45.9925
MXNet C API Concurrency 183971 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1596 0.7530 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7980 4.1450 0.0000 0.0230 0.0005
MXNDArraySetGradState 7980 4.0730 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1596 0.6830 0.0000 0.0010 0.0004
MXAutogradBackwardEx 798 442.0710 0.5250 0.9450 0.5540
MXNDArrayGetContext 12732 5.5930 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26658 685.4020 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3192 7805.1548 0.7010 3.7500 2.4452
FullyConnected 5508 531.4920 0.0080 0.3430 0.0965
argmax 1836 36.3680 0.0050 0.0480 0.0198
_backward_pick 1596 25.3620 0.0130 0.0320 0.0159
_backward_mul_scalar 1596 6.9840 0.0030 0.0220 0.0044
DeleteVariable 68160 486.3690 0.0010 0.9010 0.0071
Flatten 1836 76.9590 0.0080 0.1120 0.0419
Activation 7344 341.5680 0.0030 1.9440 0.0465
SetValueOp 1596 10.3610 0.0050 0.0180 0.0065
pick 1596 12.5710 0.0060 0.0280 0.0079
Pooling 3672 5117.6719 0.0640 3.1720 1.3937
Convolution 3672 20883.1992 0.2260 13.2050 5.6871
CopyCPU2CPU 1836 10.5540 0.0030 0.0360 0.0057
log_softmax 1596 49.7330 0.0150 0.0550 0.0312
_backward_copy 1596 82.5880 0.0220 0.0760 0.0517
_backward_Convolution 3192 29523.6113 2.6240 12.5610 9.2493
_mul_scalar 1596 5.7070 0.0020 0.0230 0.0036
_backward_log_softmax 1596 27.2550 0.0090 0.0400 0.0171
mean 5028 96.3300 0.0080 0.0730 0.0192
_backward_mean 1596 17.5490 0.0080 0.1030 0.0110
_backward_FullyConnected 4788 917.8030 0.0280 0.5940 0.1917
WaitForVar 3432 30.5060 0.0060 0.0280 0.0089
broadcast_equal 1836 9.1990 0.0030 0.0110 0.0050
sgd_update 15960 159.3230 0.0020 0.0790 0.0100
_backward_Activation 6384 472.0940 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62726 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5514 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 798 3186.2871 0.3060 73.5870 3.9928
MXNDArrayCreateEx 919 31.8300 0.0310 0.0640 0.0346
MXNDArrayWaitAll 799 17339.0449 5.6130 25.0610 21.7009
MXNDArraySyncCopyToCPU 1718 13557.6006 0.0600 24.1980 7.8915
MXNDArrayGetDType 1718 0.6190 0.0000 0.0010 0.0004
MXNDArrayGetShape 4355 2.0480 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1838 16.1690 0.0070 0.0280 0.0088
MXNDArrayFree 20537 211.2060 0.0000 0.9140 0.0103
MXNet C API Calls 92093 579.4500 487.3580 579.4500 46.0460
MXNet C API Concurrency 184185 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1598 0.7550 0.0000 0.0040 0.0005
MXNDArrayGetGradState 7990 4.1510 0.0000 0.0230 0.0005
MXNDArraySetGradState 7990 4.0770 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1598 0.6840 0.0000 0.0010 0.0004
MXAutogradBackwardEx 799 442.6180 0.5250 0.9450 0.5540
MXNDArrayGetContext 12746 5.5990 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26689 686.2250 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3196 7814.9111 0.7010 3.7500 2.4452
FullyConnected 5514 532.0350 0.0080 0.3430 0.0965
argmax 1838 36.4060 0.0050 0.0480 0.0198
_backward_pick 1598 25.3930 0.0130 0.0320 0.0159
_backward_mul_scalar 1598 6.9930 0.0030 0.0220 0.0044
DeleteVariable 68240 486.9140 0.0010 0.9010 0.0071
Flatten 1838 77.0380 0.0080 0.1120 0.0419
Activation 7352 341.9370 0.0030 1.9440 0.0465
SetValueOp 1598 10.3730 0.0050 0.0180 0.0065
pick 1598 12.5860 0.0060 0.0280 0.0079
Pooling 3676 5123.0161 0.0640 3.1720 1.3936
Convolution 3676 20905.4141 0.2260 13.2050 5.6870
CopyCPU2CPU 1838 10.5630 0.0030 0.0360 0.0057
log_softmax 1598 49.7940 0.0150 0.0550 0.0312
_backward_copy 1598 82.6930 0.0220 0.0760 0.0517
_backward_Convolution 3196 29560.4277 2.6240 12.5610 9.2492
_mul_scalar 1598 5.7140 0.0020 0.0230 0.0036
_backward_log_softmax 1598 27.2860 0.0090 0.0400 0.0171
mean 5034 96.4460 0.0080 0.0730 0.0192
_backward_mean 1598 17.5700 0.0080 0.1030 0.0110
_backward_FullyConnected 4794 918.9190 0.0280 0.5940 0.1917
WaitForVar 3436 30.5400 0.0060 0.0280 0.0089
broadcast_equal 1838 9.2090 0.0030 0.0110 0.0050
sgd_update 15980 159.4750 0.0020 0.0790 0.0100
_backward_Activation 6392 472.7450 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62800 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5520 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 799 3190.1890 0.3060 73.5870 3.9927
MXNDArrayCreateEx 920 31.8650 0.0310 0.0640 0.0346
MXNDArrayWaitAll 800 17360.7773 5.6130 25.0610 21.7010
MXNDArraySyncCopyToCPU 1720 13572.1592 0.0600 24.1980 7.8908
MXNDArrayGetDType 1720 0.6200 0.0000 0.0010 0.0004
MXNDArrayGetShape 4360 2.0510 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1840 16.1870 0.0070 0.0280 0.0088
MXNDArrayFree 20560 211.4510 0.0000 0.9140 0.0103
MXNet C API Calls 92200 579.5570 487.3580 579.5570 46.0995
MXNet C API Concurrency 184399 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1600 0.7560 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8000 4.1540 0.0000 0.0230 0.0005
MXNDArraySetGradState 8000 4.0810 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1600 0.6840 0.0000 0.0010 0.0004
MXAutogradBackwardEx 800 443.1940 0.5250 0.9450 0.5540
MXNDArrayGetContext 12760 5.6060 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26720 687.0420 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3200 7824.6431 0.7010 3.7500 2.4452
FullyConnected 5520 532.7640 0.0080 0.3430 0.0965
argmax 1840 36.4490 0.0050 0.0480 0.0198
_backward_pick 1600 25.4240 0.0130 0.0320 0.0159
_backward_mul_scalar 1600 7.0020 0.0030 0.0220 0.0044
DeleteVariable 68320 487.4560 0.0010 0.9010 0.0071
Flatten 1840 77.1150 0.0080 0.1120 0.0419
Activation 7360 342.3150 0.0030 1.9440 0.0465
SetValueOp 1600 10.3860 0.0050 0.0180 0.0065
pick 1600 12.6030 0.0060 0.0280 0.0079
Pooling 3680 5128.3662 0.0640 3.1720 1.3936
Convolution 3680 20927.6328 0.2260 13.2050 5.6869
CopyCPU2CPU 1840 10.5730 0.0030 0.0360 0.0057
log_softmax 1600 49.8660 0.0150 0.0550 0.0312
_backward_copy 1600 82.7950 0.0220 0.0760 0.0517
_backward_Convolution 3200 29597.1699 2.6240 12.5610 9.2491
_mul_scalar 1600 5.7210 0.0020 0.0230 0.0036
_backward_log_softmax 1600 27.3280 0.0090 0.0400 0.0171
mean 5040 96.5560 0.0080 0.0730 0.0192
_backward_mean 1600 17.5930 0.0080 0.1030 0.0110
_backward_FullyConnected 4800 920.3360 0.0280 0.5940 0.1917
WaitForVar 3440 30.5730 0.0060 0.0280 0.0089
broadcast_equal 1840 9.2190 0.0030 0.0110 0.0050
sgd_update 16000 159.7500 0.0020 0.0790 0.0100
_backward_Activation 6400 473.3680 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62874 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5526 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 800 3194.1169 0.3060 73.5870 3.9926
MXNDArrayCreateEx 921 31.9000 0.0310 0.0640 0.0346
MXNDArrayWaitAll 801 17382.6445 5.6130 25.0610 21.7012
MXNDArraySyncCopyToCPU 1722 13586.4092 0.0600 24.1980 7.8899
MXNDArrayGetDType 1722 0.6210 0.0000 0.0010 0.0004
MXNDArrayGetShape 4365 2.0540 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1842 16.2060 0.0070 0.0280 0.0088
MXNDArrayFree 20583 211.7120 0.0000 0.9140 0.0103
MXNet C API Calls 92307 579.6640 487.3580 579.6640 46.1530
MXNet C API Concurrency 184613 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1602 0.7580 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8010 4.1600 0.0000 0.0230 0.0005
MXNDArraySetGradState 8010 4.0880 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1602 0.6850 0.0000 0.0010 0.0004
MXAutogradBackwardEx 801 443.7430 0.5250 0.9450 0.5540
MXNDArrayGetContext 12774 5.6110 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26751 687.8600 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3204 7834.4360 0.7010 3.7500 2.4452
FullyConnected 5526 533.2860 0.0080 0.3430 0.0965
argmax 1842 36.4870 0.0050 0.0480 0.0198
_backward_pick 1602 25.4560 0.0130 0.0320 0.0159
_backward_mul_scalar 1602 7.0110 0.0030 0.0220 0.0044
DeleteVariable 68400 487.9930 0.0010 0.9010 0.0071
Flatten 1842 77.1920 0.0080 0.1120 0.0419
Activation 7368 342.6830 0.0030 1.9440 0.0465
SetValueOp 1602 10.4040 0.0050 0.0180 0.0065
pick 1602 12.6180 0.0060 0.0280 0.0079
Pooling 3684 5133.7441 0.0640 3.1720 1.3935
Convolution 3684 20949.7891 0.2260 13.2050 5.6867
CopyCPU2CPU 1842 10.5830 0.0030 0.0360 0.0057
log_softmax 1602 49.9250 0.0150 0.0550 0.0312
_backward_copy 1602 82.9000 0.0220 0.0760 0.0517
_backward_Convolution 3204 29634.2129 2.6240 12.5610 9.2491
_mul_scalar 1602 5.7280 0.0020 0.0230 0.0036
_backward_log_softmax 1602 27.3600 0.0090 0.0400 0.0171
mean 5046 96.6660 0.0080 0.0730 0.0192
_backward_mean 1602 17.6150 0.0080 0.1030 0.0110
_backward_FullyConnected 4806 921.4100 0.0280 0.5940 0.1917
WaitForVar 3444 30.6100 0.0060 0.0280 0.0089
broadcast_equal 1842 9.2280 0.0030 0.0110 0.0050
sgd_update 16020 159.9040 0.0020 0.0790 0.0100
_backward_Activation 6408 473.9930 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 62948 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5532 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 801 3198.0459 0.3060 73.5870 3.9926
MXNDArrayCreateEx 922 31.9350 0.0310 0.0640 0.0346
MXNDArrayWaitAll 802 17404.5371 5.6130 25.0610 21.7014
MXNDArraySyncCopyToCPU 1724 13600.7549 0.0600 24.1980 7.8891
MXNDArrayGetDType 1724 0.6230 0.0000 0.0010 0.0004
MXNDArrayGetShape 4370 2.0560 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1844 16.2250 0.0070 0.0280 0.0088
MXNDArrayFree 20606 211.9440 0.0000 0.9140 0.0103
MXNet C API Calls 92414 579.7710 487.3580 579.7710 46.2065
MXNet C API Concurrency 184827 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1604 0.7590 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8020 4.1650 0.0000 0.0230 0.0005
MXNDArraySetGradState 8020 4.0930 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1604 0.6850 0.0000 0.0010 0.0004
MXAutogradBackwardEx 802 444.2970 0.5250 0.9450 0.5540
MXNDArrayGetContext 12788 5.6170 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26782 688.6840 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3208 7844.2578 0.7010 3.7500 2.4452
FullyConnected 5532 533.8900 0.0080 0.3430 0.0965
argmax 1844 36.5270 0.0050 0.0480 0.0198
_backward_pick 1604 25.4870 0.0130 0.0320 0.0159
_backward_mul_scalar 1604 7.0190 0.0030 0.0220 0.0044
DeleteVariable 68480 488.5410 0.0010 0.9010 0.0071
Flatten 1844 77.2690 0.0080 0.1120 0.0419
Activation 7376 343.0520 0.0030 1.9440 0.0465
SetValueOp 1604 10.4170 0.0050 0.0180 0.0065
pick 1604 12.6330 0.0060 0.0280 0.0079
Pooling 3688 5139.1270 0.0640 3.1720 1.3935
Convolution 3688 20972.0508 0.2260 13.2050 5.6866
CopyCPU2CPU 1844 10.5930 0.0030 0.0360 0.0057
log_softmax 1604 49.9840 0.0150 0.0550 0.0312
_backward_copy 1604 83.0070 0.0220 0.0760 0.0518
_backward_Convolution 3208 29671.2812 2.6240 12.5610 9.2492
_mul_scalar 1604 5.7350 0.0020 0.0230 0.0036
_backward_log_softmax 1604 27.3920 0.0090 0.0400 0.0171
mean 5052 96.7840 0.0080 0.0730 0.0192
_backward_mean 1604 17.6370 0.0080 0.1030 0.0110
_backward_FullyConnected 4812 922.4780 0.0280 0.5940 0.1917
WaitForVar 3448 30.6460 0.0060 0.0280 0.0089
broadcast_equal 1844 9.2370 0.0030 0.0110 0.0050
sgd_update 16040 160.0500 0.0020 0.0790 0.0100
_backward_Activation 6416 474.6260 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63022 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5538 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 802 3201.9729 0.3060 73.5870 3.9925
MXNDArrayCreateEx 923 31.9710 0.0310 0.0640 0.0346
MXNDArrayWaitAll 803 17426.2363 5.6130 25.0610 21.7014
MXNDArraySyncCopyToCPU 1726 13614.8398 0.0600 24.1980 7.8881
MXNDArrayGetDType 1726 0.6240 0.0000 0.0010 0.0004
MXNDArrayGetShape 4375 2.0600 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1846 16.2440 0.0070 0.0280 0.0088
MXNDArrayFree 20629 212.1810 0.0000 0.9140 0.0103
MXNet C API Calls 92521 579.8780 487.3580 579.8780 46.2600
MXNet C API Concurrency 185041 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1606 0.7600 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8030 4.1730 0.0000 0.0230 0.0005
MXNDArraySetGradState 8030 4.0990 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1606 0.6860 0.0000 0.0010 0.0004
MXAutogradBackwardEx 803 444.8670 0.5250 0.9450 0.5540
MXNDArrayGetContext 12802 5.6220 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26813 689.5600 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3212 7854.0381 0.7010 3.7500 2.4452
FullyConnected 5538 534.4340 0.0080 0.3430 0.0965
argmax 1846 36.5660 0.0050 0.0480 0.0198
_backward_pick 1606 25.5170 0.0130 0.0320 0.0159
_backward_mul_scalar 1606 7.0280 0.0030 0.0220 0.0044
DeleteVariable 68560 489.1060 0.0010 0.9010 0.0071
Flatten 1846 77.3470 0.0080 0.1120 0.0419
Activation 7384 343.4200 0.0030 1.9440 0.0465
SetValueOp 1606 10.4300 0.0050 0.0180 0.0065
pick 1606 12.6480 0.0060 0.0280 0.0079
Pooling 3692 5144.5088 0.0640 3.1720 1.3934
Convolution 3692 20994.2344 0.2260 13.2050 5.6864
CopyCPU2CPU 1846 10.6040 0.0030 0.0360 0.0057
log_softmax 1606 50.0440 0.0150 0.0550 0.0312
_backward_copy 1606 83.1120 0.0220 0.0760 0.0518
_backward_Convolution 3212 29708.0977 2.6240 12.5610 9.2491
_mul_scalar 1606 5.7420 0.0020 0.0230 0.0036
_backward_log_softmax 1606 27.4240 0.0090 0.0400 0.0171
mean 5058 96.9060 0.0080 0.0730 0.0192
_backward_mean 1606 17.6600 0.0080 0.1030 0.0110
_backward_FullyConnected 4818 923.5500 0.0280 0.5940 0.1917
WaitForVar 3452 30.6820 0.0060 0.0280 0.0089
broadcast_equal 1846 9.2460 0.0030 0.0110 0.0050
sgd_update 16060 160.2530 0.0020 0.0790 0.0100
_backward_Activation 6424 475.1980 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63096 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5544 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 803 3205.9070 0.3060 73.5870 3.9924
MXNDArrayCreateEx 924 32.0070 0.0310 0.0640 0.0346
MXNDArrayWaitAll 804 17448.0137 5.6130 25.0610 21.7015
MXNDArraySyncCopyToCPU 1728 13629.4180 0.0600 24.1980 7.8874
MXNDArrayGetDType 1728 0.6250 0.0000 0.0010 0.0004
MXNDArrayGetShape 4380 2.0620 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1848 16.2620 0.0070 0.0280 0.0088
MXNDArrayFree 20652 212.4200 0.0000 0.9140 0.0103
MXNet C API Calls 92628 579.9850 487.3580 579.9850 46.3135
MXNet C API Concurrency 185255 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1608 0.7600 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8040 4.1790 0.0000 0.0230 0.0005
MXNDArraySetGradState 8040 4.1040 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1608 0.6860 0.0000 0.0010 0.0004
MXAutogradBackwardEx 804 445.4180 0.5250 0.9450 0.5540
MXNDArrayGetContext 12816 5.6280 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26844 690.3920 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3216 7863.8262 0.7010 3.7500 2.4452
FullyConnected 5544 535.1630 0.0080 0.3430 0.0965
argmax 1848 36.6110 0.0050 0.0480 0.0198
_backward_pick 1608 25.5510 0.0130 0.0320 0.0159
_backward_mul_scalar 1608 7.0360 0.0030 0.0220 0.0044
DeleteVariable 68640 489.6570 0.0010 0.9010 0.0071
Flatten 1848 77.4270 0.0080 0.1120 0.0419
Activation 7392 343.8360 0.0030 1.9440 0.0465
SetValueOp 1608 10.4430 0.0050 0.0180 0.0065
pick 1608 12.6640 0.0060 0.0280 0.0079
Pooling 3696 5149.8721 0.0640 3.1720 1.3934
Convolution 3696 21016.4473 0.2260 13.2050 5.6863
CopyCPU2CPU 1848 10.6130 0.0030 0.0360 0.0057
log_softmax 1608 50.1170 0.0150 0.0550 0.0312
_backward_copy 1608 83.2150 0.0220 0.0760 0.0518
_backward_Convolution 3216 29744.9180 2.6240 12.5610 9.2490
_mul_scalar 1608 5.7490 0.0020 0.0230 0.0036
_backward_log_softmax 1608 27.4660 0.0090 0.0400 0.0171
mean 5064 97.0170 0.0080 0.0730 0.0192
_backward_mean 1608 17.6820 0.0080 0.1030 0.0110
_backward_FullyConnected 4824 924.9630 0.0280 0.5940 0.1917
WaitForVar 3456 30.7170 0.0060 0.0280 0.0089
broadcast_equal 1848 9.2560 0.0030 0.0110 0.0050
sgd_update 16080 160.5280 0.0020 0.0790 0.0100
_backward_Activation 6432 475.8060 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63170 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5550 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 804 3209.8320 0.3060 73.5870 3.9923
MXNDArrayCreateEx 925 32.0420 0.0310 0.0640 0.0346
MXNDArrayWaitAll 805 17469.8203 5.6130 25.0610 21.7016
MXNDArraySyncCopyToCPU 1730 13643.6025 0.0600 24.1980 7.8865
MXNDArrayGetDType 1730 0.6250 0.0000 0.0010 0.0004
MXNDArrayGetShape 4385 2.0650 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1850 16.2810 0.0070 0.0280 0.0088
MXNDArrayFree 20675 212.6620 0.0000 0.9140 0.0103
MXNet C API Calls 92735 580.0920 487.3580 580.0920 46.3670
MXNet C API Concurrency 185469 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1610 0.7600 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8050 4.1830 0.0000 0.0230 0.0005
MXNDArraySetGradState 8050 4.1090 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1610 0.6860 0.0000 0.0010 0.0004
MXAutogradBackwardEx 805 445.9670 0.5250 0.9450 0.5540
MXNDArrayGetContext 12830 5.6360 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26875 691.2270 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3220 7873.5532 0.7010 3.7500 2.4452
FullyConnected 5550 535.6860 0.0080 0.3430 0.0965
argmax 1850 36.6500 0.0050 0.0480 0.0198
_backward_pick 1610 25.5820 0.0130 0.0320 0.0159
_backward_mul_scalar 1610 7.0450 0.0030 0.0220 0.0044
DeleteVariable 68720 490.2270 0.0010 0.9010 0.0071
Flatten 1850 77.5060 0.0080 0.1120 0.0419
Activation 7400 344.2060 0.0030 1.9440 0.0465
SetValueOp 1610 10.4570 0.0050 0.0180 0.0065
pick 1610 12.6790 0.0060 0.0280 0.0079
Pooling 3700 5155.2769 0.0640 3.1720 1.3933
Convolution 3700 21038.6094 0.2260 13.2050 5.6861
CopyCPU2CPU 1850 10.6230 0.0030 0.0360 0.0057
log_softmax 1610 50.1810 0.0150 0.0550 0.0312
_backward_copy 1610 83.3200 0.0220 0.0760 0.0518
_backward_Convolution 3220 29781.8887 2.6240 12.5610 9.2490
_mul_scalar 1610 5.7560 0.0020 0.0230 0.0036
_backward_log_softmax 1610 27.4970 0.0090 0.0400 0.0171
mean 5070 97.1330 0.0080 0.0730 0.0192
_backward_mean 1610 17.7040 0.0080 0.1030 0.0110
_backward_FullyConnected 4830 926.0300 0.0280 0.5940 0.1917
WaitForVar 3460 30.7530 0.0060 0.0280 0.0089
broadcast_equal 1850 9.2650 0.0030 0.0110 0.0050
sgd_update 16100 160.7300 0.0020 0.0790 0.0100
_backward_Activation 6440 476.3790 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63244 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5556 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 805 3213.7571 0.3060 73.5870 3.9922
MXNDArrayCreateEx 926 32.0780 0.0310 0.0640 0.0346
MXNDArrayWaitAll 806 17491.6855 5.6130 25.0610 21.7018
MXNDArraySyncCopyToCPU 1732 13657.9258 0.0600 24.1980 7.8856
MXNDArrayGetDType 1732 0.6270 0.0000 0.0010 0.0004
MXNDArrayGetShape 4390 2.0660 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1852 16.2990 0.0070 0.0280 0.0088
MXNDArrayFree 20698 212.8920 0.0000 0.9140 0.0103
MXNet C API Calls 92842 580.1990 487.3580 580.1990 46.4205
MXNet C API Concurrency 185683 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1612 0.7610 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8060 4.1880 0.0000 0.0230 0.0005
MXNDArraySetGradState 8060 4.1160 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1612 0.6870 0.0000 0.0010 0.0004
MXAutogradBackwardEx 806 446.5120 0.5250 0.9450 0.5540
MXNDArrayGetContext 12844 5.6400 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26906 692.0460 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3224 7883.2720 0.7010 3.7500 2.4452
FullyConnected 5556 536.2300 0.0080 0.3430 0.0965
argmax 1852 36.6880 0.0050 0.0480 0.0198
_backward_pick 1612 25.6120 0.0130 0.0320 0.0159
_backward_mul_scalar 1612 7.0550 0.0030 0.0220 0.0044
DeleteVariable 68800 490.7510 0.0010 0.9010 0.0071
Flatten 1852 77.5850 0.0080 0.1120 0.0419
Activation 7408 344.5750 0.0030 1.9440 0.0465
SetValueOp 1612 10.4700 0.0050 0.0180 0.0065
pick 1612 12.6940 0.0060 0.0280 0.0079
Pooling 3704 5160.6318 0.0640 3.1720 1.3933
Convolution 3704 21060.8828 0.2260 13.2050 5.6860
CopyCPU2CPU 1852 10.6330 0.0030 0.0360 0.0057
log_softmax 1612 50.2400 0.0150 0.0550 0.0312
_backward_copy 1612 83.4240 0.0220 0.0760 0.0518
_backward_Convolution 3224 29818.9863 2.6240 12.5610 9.2491
_mul_scalar 1612 5.7630 0.0020 0.0230 0.0036
_backward_log_softmax 1612 27.5300 0.0090 0.0400 0.0171
mean 5076 97.2520 0.0080 0.0730 0.0192
_backward_mean 1612 17.7250 0.0080 0.1030 0.0110
_backward_FullyConnected 4836 927.1020 0.0280 0.5940 0.1917
WaitForVar 3464 30.8170 0.0060 0.0280 0.0089
broadcast_equal 1852 9.2750 0.0030 0.0110 0.0050
sgd_update 16120 160.9340 0.0020 0.0790 0.0100
_backward_Activation 6448 476.9470 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63318 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5562 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 806 3217.6909 0.3060 73.5870 3.9922
MXNDArrayCreateEx 927 32.1140 0.0310 0.0640 0.0346
MXNDArrayWaitAll 807 17513.3867 5.6130 25.0610 21.7018
MXNDArraySyncCopyToCPU 1734 13672.0176 0.0600 24.1980 7.8847
MXNDArrayGetDType 1734 0.6280 0.0000 0.0010 0.0004
MXNDArrayGetShape 4395 2.0710 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1854 16.3170 0.0070 0.0280 0.0088
MXNDArrayFree 20721 213.1320 0.0000 0.9140 0.0103
MXNet C API Calls 92949 580.3060 487.3580 580.3060 46.4740
MXNet C API Concurrency 185897 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1614 0.7630 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8070 4.1940 0.0000 0.0230 0.0005
MXNDArraySetGradState 8070 4.1210 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1614 0.6880 0.0000 0.0010 0.0004
MXAutogradBackwardEx 807 447.0820 0.5250 0.9450 0.5540
MXNDArrayGetContext 12858 5.6460 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26937 692.9190 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3228 7892.9399 0.7010 3.7500 2.4451
FullyConnected 5562 536.7730 0.0080 0.3430 0.0965
argmax 1854 36.7260 0.0050 0.0480 0.0198
_backward_pick 1614 25.6650 0.0130 0.0320 0.0159
_backward_mul_scalar 1614 7.0640 0.0030 0.0220 0.0044
DeleteVariable 68880 491.2960 0.0010 0.9010 0.0071
Flatten 1854 77.6640 0.0080 0.1120 0.0419
Activation 7416 344.9610 0.0030 1.9440 0.0465
SetValueOp 1614 10.4820 0.0050 0.0180 0.0065
pick 1614 12.7090 0.0060 0.0280 0.0079
Pooling 3708 5166.0020 0.0640 3.1720 1.3932
Convolution 3708 21083.0605 0.2260 13.2050 5.6858
CopyCPU2CPU 1854 10.6430 0.0030 0.0360 0.0057
log_softmax 1614 50.3000 0.0150 0.0550 0.0312
_backward_copy 1614 83.5290 0.0220 0.0760 0.0518
_backward_Convolution 3228 29855.7852 2.6240 12.5610 9.2490
_mul_scalar 1614 5.7700 0.0020 0.0230 0.0036
_backward_log_softmax 1614 27.5620 0.0090 0.0400 0.0171
mean 5082 97.3690 0.0080 0.0730 0.0192
_backward_mean 1614 17.7460 0.0080 0.1030 0.0110
_backward_FullyConnected 4842 928.1790 0.0280 0.5940 0.1917
WaitForVar 3468 30.8520 0.0060 0.0280 0.0089
broadcast_equal 1854 9.2850 0.0030 0.0110 0.0050
sgd_update 16140 161.1400 0.0020 0.0790 0.0100
_backward_Activation 6456 477.5160 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63392 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5568 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 807 3221.6189 0.3060 73.5870 3.9921
MXNDArrayCreateEx 928 32.1500 0.0310 0.0640 0.0346
MXNDArrayWaitAll 808 17535.1484 5.6130 25.0610 21.7019
MXNDArraySyncCopyToCPU 1736 13686.6426 0.0600 24.1980 7.8840
MXNDArrayGetDType 1736 0.6290 0.0000 0.0010 0.0004
MXNDArrayGetShape 4400 2.0730 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1856 16.3360 0.0070 0.0280 0.0088
MXNDArrayFree 20744 213.3710 0.0000 0.9140 0.0103
MXNet C API Calls 93056 580.4130 487.3580 580.4130 46.5275
MXNet C API Concurrency 186111 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1616 0.7630 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8080 4.1960 0.0000 0.0230 0.0005
MXNDArraySetGradState 8080 4.1270 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1616 0.6880 0.0000 0.0010 0.0004
MXAutogradBackwardEx 808 447.6290 0.5250 0.9450 0.5540
MXNDArrayGetContext 12872 5.6540 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26968 693.7340 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3232 7902.7578 0.7010 3.7500 2.4452
FullyConnected 5568 537.5030 0.0080 0.3430 0.0965
argmax 1856 36.7700 0.0050 0.0480 0.0198
_backward_pick 1616 25.6980 0.0130 0.0320 0.0159
_backward_mul_scalar 1616 7.0730 0.0030 0.0220 0.0044
DeleteVariable 68960 491.8480 0.0010 0.9010 0.0071
Flatten 1856 77.7420 0.0080 0.1120 0.0419
Activation 7424 345.3380 0.0030 1.9440 0.0465
SetValueOp 1616 10.4950 0.0050 0.0180 0.0065
pick 1616 12.7260 0.0060 0.0280 0.0079
Pooling 3712 5171.3389 0.0640 3.1720 1.3931
Convolution 3712 21105.2754 0.2260 13.2050 5.6857
CopyCPU2CPU 1856 10.6520 0.0030 0.0360 0.0057
log_softmax 1616 50.3730 0.0150 0.0550 0.0312
_backward_copy 1616 83.6300 0.0220 0.0760 0.0518
_backward_Convolution 3232 29892.5742 2.6240 12.5610 9.2489
_mul_scalar 1616 5.7770 0.0020 0.0230 0.0036
_backward_log_softmax 1616 27.6040 0.0090 0.0400 0.0171
mean 5088 97.4790 0.0080 0.0730 0.0192
_backward_mean 1616 17.7690 0.0080 0.1030 0.0110
_backward_FullyConnected 4848 929.6410 0.0280 0.5940 0.1918
WaitForVar 3472 30.8880 0.0060 0.0280 0.0089
broadcast_equal 1856 9.2950 0.0030 0.0110 0.0050
sgd_update 16160 161.4180 0.0020 0.0790 0.0100
_backward_Activation 6464 478.1180 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63466 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5574 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 808 3225.5610 0.3060 73.5870 3.9920
MXNDArrayCreateEx 929 32.1860 0.0310 0.0640 0.0346
MXNDArrayWaitAll 809 17556.9863 5.6130 25.0610 21.7021
MXNDArraySyncCopyToCPU 1738 13700.7471 0.0600 24.1980 7.8831
MXNDArrayGetDType 1738 0.6290 0.0000 0.0010 0.0004
MXNDArrayGetShape 4405 2.0770 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1858 16.3550 0.0070 0.0280 0.0088
MXNDArrayFree 20767 213.6120 0.0000 0.9140 0.0103
MXNet C API Calls 93163 580.5200 487.3580 580.5200 46.5810
MXNet C API Concurrency 186325 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1618 0.7650 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8090 4.2010 0.0000 0.0230 0.0005
MXNDArraySetGradState 8090 4.1330 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1618 0.6900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 809 448.2020 0.5250 0.9450 0.5540
MXNDArrayGetContext 12886 5.6650 0.0000 0.0190 0.0004
MXImperativeInvokeEx 26999 694.5850 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3236 7912.5132 0.7010 3.7500 2.4452
FullyConnected 5574 538.0260 0.0080 0.3430 0.0965
argmax 1858 36.8080 0.0050 0.0480 0.0198
_backward_pick 1618 25.7280 0.0130 0.0320 0.0159
_backward_mul_scalar 1618 7.0820 0.0030 0.0220 0.0044
DeleteVariable 69040 492.4060 0.0010 0.9010 0.0071
Flatten 1858 77.8220 0.0080 0.1120 0.0419
Activation 7432 345.7070 0.0030 1.9440 0.0465
SetValueOp 1618 10.5070 0.0050 0.0180 0.0065
pick 1618 12.7410 0.0060 0.0280 0.0079
Pooling 3716 5176.7720 0.0640 3.1720 1.3931
Convolution 3716 21127.4238 0.2260 13.2050 5.6855
CopyCPU2CPU 1858 10.6610 0.0030 0.0360 0.0057
log_softmax 1618 50.4330 0.0150 0.0550 0.0312
_backward_copy 1618 83.7360 0.0220 0.0760 0.0518
_backward_Convolution 3236 29929.6094 2.6240 12.5610 9.2490
_mul_scalar 1618 5.7840 0.0020 0.0230 0.0036
_backward_log_softmax 1618 27.6360 0.0090 0.0400 0.0171
mean 5094 97.5980 0.0080 0.0730 0.0192
_backward_mean 1618 17.7900 0.0080 0.1030 0.0110
_backward_FullyConnected 4854 930.7120 0.0280 0.5940 0.1917
WaitForVar 3476 30.9230 0.0060 0.0280 0.0089
broadcast_equal 1858 9.3050 0.0030 0.0110 0.0050
sgd_update 16180 161.6240 0.0020 0.0790 0.0100
_backward_Activation 6472 478.6660 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63540 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5580 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 809 3229.4961 0.3060 73.5870 3.9920
MXNDArrayCreateEx 930 32.2230 0.0310 0.0640 0.0346
MXNDArrayWaitAll 810 17578.9043 5.6130 25.0610 21.7024
MXNDArraySyncCopyToCPU 1740 13714.6885 0.0600 24.1980 7.8820
MXNDArrayGetDType 1740 0.6300 0.0000 0.0010 0.0004
MXNDArrayGetShape 4410 2.0790 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1860 16.3740 0.0070 0.0280 0.0088
MXNDArrayFree 20790 213.8450 0.0000 0.9140 0.0103
MXNet C API Calls 93270 580.6270 487.3580 580.6270 46.6345
MXNet C API Concurrency 186539 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1620 0.7660 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8100 4.2070 0.0000 0.0230 0.0005
MXNDArraySetGradState 8100 4.1370 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1620 0.6910 0.0000 0.0010 0.0004
MXAutogradBackwardEx 810 448.7730 0.5250 0.9450 0.5540
MXNDArrayGetContext 12900 5.6700 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27030 695.4480 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3240 7922.2422 0.7010 3.7500 2.4451
FullyConnected 5580 538.5460 0.0080 0.3430 0.0965
argmax 1860 36.8450 0.0050 0.0480 0.0198
_backward_pick 1620 25.7580 0.0130 0.0320 0.0159
_backward_mul_scalar 1620 7.0900 0.0030 0.0220 0.0044
DeleteVariable 69120 492.9390 0.0010 0.9010 0.0071
Flatten 1860 77.9030 0.0080 0.1120 0.0419
Activation 7440 346.0740 0.0030 1.9440 0.0465
SetValueOp 1620 10.5220 0.0050 0.0180 0.0065
pick 1620 12.7560 0.0060 0.0280 0.0079
Pooling 3720 5182.1182 0.0640 3.1720 1.3930
Convolution 3720 21149.7188 0.2260 13.2050 5.6854
CopyCPU2CPU 1860 10.6710 0.0030 0.0360 0.0057
log_softmax 1620 50.4930 0.0150 0.0550 0.0312
_backward_copy 1620 83.8830 0.0220 0.0760 0.0518
_backward_Convolution 3240 29966.7676 2.6240 12.5610 9.2490
_mul_scalar 1620 5.7900 0.0020 0.0230 0.0036
_backward_log_softmax 1620 27.6680 0.0090 0.0400 0.0171
mean 5100 97.7180 0.0080 0.0730 0.0192
_backward_mean 1620 17.8120 0.0080 0.1030 0.0110
_backward_FullyConnected 4860 932.1260 0.0280 0.5940 0.1918
WaitForVar 3480 30.9600 0.0060 0.0280 0.0089
broadcast_equal 1860 9.3140 0.0030 0.0110 0.0050
sgd_update 16200 161.8310 0.0020 0.0790 0.0100
_backward_Activation 6480 479.2140 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63614 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5586 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 810 3233.4180 0.3060 73.5870 3.9919
MXNDArrayCreateEx 931 32.2590 0.0310 0.0640 0.0346
MXNDArrayWaitAll 811 17600.7012 5.6130 25.0610 21.7025
MXNDArraySyncCopyToCPU 1742 13728.9268 0.0600 24.1980 7.8811
MXNDArrayGetDType 1742 0.6300 0.0000 0.0010 0.0004
MXNDArrayGetShape 4415 2.0820 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1862 16.3930 0.0070 0.0280 0.0088
MXNDArrayFree 20813 214.0810 0.0000 0.9140 0.0103
MXNet C API Calls 93377 580.7340 487.3580 580.7340 46.6880
MXNet C API Concurrency 186753 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1622 0.7670 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8110 4.2130 0.0000 0.0230 0.0005
MXNDArraySetGradState 8110 4.1430 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1622 0.6930 0.0000 0.0010 0.0004
MXAutogradBackwardEx 811 449.3430 0.5250 0.9450 0.5541
MXNDArrayGetContext 12914 5.6770 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27061 696.2730 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3244 7932.0288 0.7010 3.7500 2.4451
FullyConnected 5586 539.0690 0.0080 0.3430 0.0965
argmax 1862 36.8820 0.0050 0.0480 0.0198
_backward_pick 1622 25.7890 0.0130 0.0320 0.0159
_backward_mul_scalar 1622 7.0980 0.0030 0.0220 0.0044
DeleteVariable 69200 493.4790 0.0010 0.9010 0.0071
Flatten 1862 77.9810 0.0080 0.1120 0.0419
Activation 7448 346.4440 0.0030 1.9440 0.0465
SetValueOp 1622 10.5350 0.0050 0.0180 0.0065
pick 1622 12.7710 0.0060 0.0280 0.0079
Pooling 3724 5187.5562 0.0640 3.1720 1.3930
Convolution 3724 21171.8730 0.2260 13.2050 5.6853
CopyCPU2CPU 1862 10.6810 0.0030 0.0360 0.0057
log_softmax 1622 50.5530 0.0150 0.0550 0.0312
_backward_copy 1622 83.9890 0.0220 0.0760 0.0518
_backward_Convolution 3244 30003.6074 2.6240 12.5610 9.2490
_mul_scalar 1622 5.7970 0.0020 0.0230 0.0036
_backward_log_softmax 1622 27.7000 0.0090 0.0400 0.0171
mean 5106 97.8400 0.0080 0.0730 0.0192
_backward_mean 1622 17.8330 0.0080 0.1030 0.0110
_backward_FullyConnected 4866 933.2010 0.0280 0.5940 0.1918
WaitForVar 3484 30.9950 0.0060 0.0280 0.0089
broadcast_equal 1862 9.3250 0.0030 0.0110 0.0050
sgd_update 16220 162.0360 0.0020 0.0790 0.0100
_backward_Activation 6488 479.7940 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63688 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5592 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 811 3237.3401 0.3060 73.5870 3.9918
MXNDArrayCreateEx 932 32.2950 0.0310 0.0640 0.0347
MXNDArrayWaitAll 812 17622.3965 5.6130 25.0610 21.7025
MXNDArraySyncCopyToCPU 1744 13743.5703 0.0600 24.1980 7.8805
MXNDArrayGetDType 1744 0.6300 0.0000 0.0010 0.0004
MXNDArrayGetShape 4420 2.0840 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1864 16.4110 0.0070 0.0280 0.0088
MXNDArrayFree 20836 214.3140 0.0000 0.9140 0.0103
MXNet C API Calls 93484 580.8410 487.3580 580.8410 46.7415
MXNet C API Concurrency 186967 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1624 0.7680 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8120 4.2190 0.0000 0.0230 0.0005
MXNDArraySetGradState 8120 4.1490 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1624 0.6930 0.0000 0.0010 0.0004
MXAutogradBackwardEx 812 449.8870 0.5250 0.9450 0.5540
MXNDArrayGetContext 12928 5.6840 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27092 697.0890 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3248 7941.7549 0.7010 3.7500 2.4451
FullyConnected 5592 539.8040 0.0080 0.3430 0.0965
argmax 1864 36.9260 0.0050 0.0480 0.0198
_backward_pick 1624 25.8220 0.0130 0.0320 0.0159
_backward_mul_scalar 1624 7.1070 0.0030 0.0220 0.0044
DeleteVariable 69280 494.0140 0.0010 0.9010 0.0071
Flatten 1864 78.0590 0.0080 0.1120 0.0419
Activation 7456 346.8190 0.0030 1.9440 0.0465
SetValueOp 1624 10.5490 0.0050 0.0180 0.0065
pick 1624 12.7880 0.0060 0.0280 0.0079
Pooling 3728 5192.9531 0.0640 3.1720 1.3930
Convolution 3728 21194.0742 0.2260 13.2050 5.6851
CopyCPU2CPU 1864 10.6910 0.0030 0.0360 0.0057
log_softmax 1624 50.6250 0.0150 0.0550 0.0312
_backward_copy 1624 84.0910 0.0220 0.0760 0.0518
_backward_Convolution 3248 30040.3262 2.6240 12.5610 9.2489
_mul_scalar 1624 5.8040 0.0020 0.0230 0.0036
_backward_log_softmax 1624 27.7410 0.0090 0.0400 0.0171
mean 5112 97.9490 0.0080 0.0730 0.0192
_backward_mean 1624 17.8550 0.0080 0.1030 0.0110
_backward_FullyConnected 4872 934.6170 0.0280 0.5940 0.1918
WaitForVar 3488 31.0310 0.0060 0.0280 0.0089
broadcast_equal 1864 9.3340 0.0030 0.0110 0.0050
sgd_update 16240 162.3130 0.0020 0.0790 0.0100
_backward_Activation 6496 480.3990 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63762 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5598 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 812 3241.2351 0.3060 73.5870 3.9917
MXNDArrayCreateEx 933 32.3300 0.0310 0.0640 0.0347
MXNDArrayWaitAll 813 17644.3047 5.6130 25.0610 21.7027
MXNDArraySyncCopyToCPU 1746 13757.8232 0.0600 24.1980 7.8796
MXNDArrayGetDType 1746 0.6310 0.0000 0.0010 0.0004
MXNDArrayGetShape 4425 2.0870 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1866 16.4290 0.0070 0.0280 0.0088
MXNDArrayFree 20859 214.5520 0.0000 0.9140 0.0103
MXNet C API Calls 93591 580.9480 487.3580 580.9480 46.7950
MXNet C API Concurrency 187181 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1626 0.7700 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8130 4.2230 0.0000 0.0230 0.0005
MXNDArraySetGradState 8130 4.1550 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1626 0.6940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 813 450.4300 0.5250 0.9450 0.5540
MXNDArrayGetContext 12942 5.6930 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27123 697.9050 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3252 7951.5181 0.7010 3.7500 2.4451
FullyConnected 5598 540.3260 0.0080 0.3430 0.0965
argmax 1866 36.9640 0.0050 0.0480 0.0198
_backward_pick 1626 25.8540 0.0130 0.0320 0.0159
_backward_mul_scalar 1626 7.1150 0.0030 0.0220 0.0044
DeleteVariable 69360 494.5690 0.0010 0.9010 0.0071
Flatten 1866 78.1380 0.0080 0.1120 0.0419
Activation 7464 347.1880 0.0030 1.9440 0.0465
SetValueOp 1626 10.5610 0.0050 0.0180 0.0065
pick 1626 12.8040 0.0060 0.0280 0.0079
Pooling 3732 5198.3340 0.0640 3.1720 1.3929
Convolution 3732 21216.2578 0.2260 13.2050 5.6850
CopyCPU2CPU 1866 10.7010 0.0030 0.0360 0.0057
log_softmax 1626 50.6840 0.0150 0.0550 0.0312
_backward_copy 1626 84.1980 0.0220 0.0760 0.0518
_backward_Convolution 3252 30077.4062 2.6240 12.5610 9.2489
_mul_scalar 1626 5.8110 0.0020 0.0230 0.0036
_backward_log_softmax 1626 27.7730 0.0090 0.0400 0.0171
mean 5118 98.0640 0.0080 0.0730 0.0192
_backward_mean 1626 17.8770 0.0080 0.1030 0.0110
_backward_FullyConnected 4878 935.6850 0.0280 0.5940 0.1918
WaitForVar 3492 31.0670 0.0060 0.0280 0.0089
broadcast_equal 1866 9.3450 0.0030 0.0110 0.0050
sgd_update 16260 162.5190 0.0020 0.0790 0.0100
_backward_Activation 6504 480.9830 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63836 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5604 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 813 3245.1790 0.3060 73.5870 3.9916
MXNDArrayCreateEx 934 32.3660 0.0310 0.0640 0.0347
MXNDArrayWaitAll 814 17666.1621 5.6130 25.0610 21.7029
MXNDArraySyncCopyToCPU 1748 13772.0938 0.0600 24.1980 7.8788
MXNDArrayGetDType 1748 0.6330 0.0000 0.0010 0.0004
MXNDArrayGetShape 4430 2.0880 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1868 16.4470 0.0070 0.0280 0.0088
MXNDArrayFree 20882 214.7950 0.0000 0.9140 0.0103
MXNet C API Calls 93698 581.0550 487.3580 581.0550 46.8485
MXNet C API Concurrency 187395 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1628 0.7710 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8140 4.2260 0.0000 0.0230 0.0005
MXNDArraySetGradState 8140 4.1600 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1628 0.6960 0.0000 0.0010 0.0004
MXAutogradBackwardEx 814 450.9850 0.5250 0.9450 0.5540
MXNDArrayGetContext 12956 5.7000 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27154 698.7370 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3256 7961.2988 0.7010 3.7500 2.4451
FullyConnected 5604 540.8470 0.0080 0.3430 0.0965
argmax 1868 37.0010 0.0050 0.0480 0.0198
_backward_pick 1628 25.8850 0.0130 0.0320 0.0159
_backward_mul_scalar 1628 7.1240 0.0030 0.0220 0.0044
DeleteVariable 69440 495.1430 0.0010 0.9010 0.0071
Flatten 1868 78.2190 0.0080 0.1120 0.0419
Activation 7472 347.5550 0.0030 1.9440 0.0465
SetValueOp 1628 10.5750 0.0050 0.0180 0.0065
pick 1628 12.8190 0.0060 0.0280 0.0079
Pooling 3736 5203.7119 0.0640 3.1720 1.3929
Convolution 3736 21238.4980 0.2260 13.2050 5.6848
CopyCPU2CPU 1868 10.7100 0.0030 0.0360 0.0057
log_softmax 1628 50.7430 0.0150 0.0550 0.0312
_backward_copy 1628 84.3050 0.0220 0.0760 0.0518
_backward_Convolution 3256 30114.4531 2.6240 12.5610 9.2489
_mul_scalar 1628 5.8180 0.0020 0.0230 0.0036
_backward_log_softmax 1628 27.8050 0.0090 0.0400 0.0171
mean 5124 98.1800 0.0080 0.0730 0.0192
_backward_mean 1628 17.8980 0.0080 0.1030 0.0110
_backward_FullyConnected 4884 936.7560 0.0280 0.5940 0.1918
WaitForVar 3496 31.1020 0.0060 0.0280 0.0089
broadcast_equal 1868 9.3540 0.0030 0.0110 0.0050
sgd_update 16280 162.6690 0.0020 0.0790 0.0100
_backward_Activation 6512 481.6220 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63910 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5610 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 814 3249.1260 0.3060 73.5870 3.9916
MXNDArrayCreateEx 935 32.4020 0.0310 0.0640 0.0347
MXNDArrayWaitAll 815 17687.9238 5.6130 25.0610 21.7030
MXNDArraySyncCopyToCPU 1750 13786.3701 0.0600 24.1980 7.8779
MXNDArrayGetDType 1750 0.6340 0.0000 0.0010 0.0004
MXNDArrayGetShape 4435 2.0890 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1870 16.4660 0.0070 0.0280 0.0088
MXNDArrayFree 20905 215.0340 0.0000 0.9140 0.0103
MXNet C API Calls 93805 581.1620 487.3580 581.1620 46.9020
MXNet C API Concurrency 187609 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1630 0.7720 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8150 4.2310 0.0000 0.0230 0.0005
MXNDArraySetGradState 8150 4.1650 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1630 0.6960 0.0000 0.0010 0.0004
MXAutogradBackwardEx 815 451.5350 0.5250 0.9450 0.5540
MXNDArrayGetContext 12970 5.7040 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27185 699.5610 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3260 7971.0918 0.7010 3.7500 2.4451
FullyConnected 5610 541.3660 0.0080 0.3430 0.0965
argmax 1870 37.0400 0.0050 0.0480 0.0198
_backward_pick 1630 25.9180 0.0130 0.0320 0.0159
_backward_mul_scalar 1630 7.1330 0.0030 0.0220 0.0044
DeleteVariable 69520 495.6840 0.0010 0.9010 0.0071
Flatten 1870 78.2980 0.0080 0.1120 0.0419
Activation 7480 347.9280 0.0030 1.9440 0.0465
SetValueOp 1630 10.5880 0.0050 0.0180 0.0065
pick 1630 12.8330 0.0060 0.0280 0.0079
Pooling 3740 5209.0811 0.0640 3.1720 1.3928
Convolution 3740 21260.7207 0.2260 13.2050 5.6847
CopyCPU2CPU 1870 10.7200 0.0030 0.0360 0.0057
log_softmax 1630 50.8030 0.0150 0.0550 0.0312
_backward_copy 1630 84.4100 0.0220 0.0760 0.0518
_backward_Convolution 3260 30151.2793 2.6240 12.5610 9.2489
_mul_scalar 1630 5.8250 0.0020 0.0230 0.0036
_backward_log_softmax 1630 27.8370 0.0090 0.0400 0.0171
mean 5130 98.3000 0.0080 0.0730 0.0192
_backward_mean 1630 17.9200 0.0080 0.1030 0.0110
_backward_FullyConnected 4890 937.8260 0.0280 0.5940 0.1918
WaitForVar 3500 31.1380 0.0060 0.0280 0.0089
broadcast_equal 1870 9.3640 0.0030 0.0110 0.0050
sgd_update 16300 162.8790 0.0020 0.0790 0.0100
_backward_Activation 6520 482.1900 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 63984 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5616 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 815 3253.0520 0.3060 73.5870 3.9915
MXNDArrayCreateEx 936 32.4370 0.0310 0.0640 0.0347
MXNDArrayWaitAll 816 17709.6191 5.6130 25.0610 21.7030
MXNDArraySyncCopyToCPU 1752 13800.9492 0.0600 24.1980 7.8773
MXNDArrayGetDType 1752 0.6340 0.0000 0.0010 0.0004
MXNDArrayGetShape 4440 2.0890 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1872 16.4830 0.0070 0.0280 0.0088
MXNDArrayFree 20928 215.2680 0.0000 0.9140 0.0103
MXNet C API Calls 93912 581.2690 487.3580 581.2690 46.9555
MXNet C API Concurrency 187823 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1632 0.7730 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8160 4.2350 0.0000 0.0230 0.0005
MXNDArraySetGradState 8160 4.1690 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1632 0.6970 0.0000 0.0010 0.0004
MXAutogradBackwardEx 816 452.0880 0.5250 0.9450 0.5540
MXNDArrayGetContext 12984 5.7090 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27216 700.3880 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3264 7980.7852 0.7010 3.7500 2.4451
FullyConnected 5616 542.1100 0.0080 0.3430 0.0965
argmax 1872 37.0850 0.0050 0.0480 0.0198
_backward_pick 1632 25.9500 0.0130 0.0320 0.0159
_backward_mul_scalar 1632 7.1410 0.0030 0.0220 0.0044
DeleteVariable 69600 496.2350 0.0010 0.9010 0.0071
Flatten 1872 78.3760 0.0080 0.1120 0.0419
Activation 7488 348.3070 0.0030 1.9440 0.0465
SetValueOp 1632 10.6010 0.0050 0.0180 0.0065
pick 1632 12.8500 0.0060 0.0280 0.0079
Pooling 3744 5214.4268 0.0640 3.1720 1.3927
Convolution 3744 21282.9414 0.2260 13.2050 5.6845
CopyCPU2CPU 1872 10.7290 0.0030 0.0360 0.0057
log_softmax 1632 50.8740 0.0150 0.0550 0.0312
_backward_copy 1632 84.5130 0.0220 0.0760 0.0518
_backward_Convolution 3264 30188.0762 2.6240 12.5610 9.2488
_mul_scalar 1632 5.8320 0.0020 0.0230 0.0036
_backward_log_softmax 1632 27.8800 0.0090 0.0400 0.0171
mean 5136 98.4100 0.0080 0.0730 0.0192
_backward_mean 1632 17.9420 0.0080 0.1030 0.0110
_backward_FullyConnected 4896 939.2280 0.0280 0.5940 0.1918
WaitForVar 3504 31.1730 0.0060 0.0280 0.0089
broadcast_equal 1872 9.3730 0.0030 0.0110 0.0050
sgd_update 16320 163.1590 0.0020 0.0790 0.0100
_backward_Activation 6528 482.7640 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64058 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5622 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 816 3253.7791 0.3060 73.5870 3.9875
MXNDArrayCreateEx 937 32.4740 0.0310 0.0640 0.0347
MXNDArrayWaitAll 817 17731.4590 5.6130 25.0610 21.7031
MXNDArraySyncCopyToCPU 1754 13815.1729 0.0600 24.1980 7.8764
MXNDArrayGetDType 1754 0.6360 0.0000 0.0010 0.0004
MXNDArrayGetShape 4445 2.0910 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1874 16.5010 0.0070 0.0280 0.0088
MXNDArrayFree 20951 215.5090 0.0000 0.9140 0.0103
MXNet C API Calls 94019 581.3760 487.3580 581.3760 47.0090
MXNet C API Concurrency 188037 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1634 0.7730 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8170 4.2410 0.0000 0.0230 0.0005
MXNDArraySetGradState 8170 4.1750 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1634 0.6980 0.0000 0.0010 0.0004
MXAutogradBackwardEx 817 452.6370 0.5250 0.9450 0.5540
MXNDArrayGetContext 12998 5.7130 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27247 701.2180 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3268 7990.4819 0.7010 3.7500 2.4451
FullyConnected 5622 542.6300 0.0080 0.3430 0.0965
argmax 1874 37.1240 0.0050 0.0480 0.0198
_backward_pick 1634 25.9800 0.0130 0.0320 0.0159
_backward_mul_scalar 1634 7.1500 0.0030 0.0220 0.0044
DeleteVariable 69680 496.8020 0.0010 0.9010 0.0071
Flatten 1874 78.4540 0.0080 0.1120 0.0419
Activation 7496 348.6770 0.0030 1.9440 0.0465
SetValueOp 1634 10.6140 0.0050 0.0180 0.0065
pick 1634 12.8650 0.0060 0.0280 0.0079
Pooling 3748 5219.8169 0.0640 3.1720 1.3927
Convolution 3748 21305.1348 0.2260 13.2050 5.6844
CopyCPU2CPU 1874 10.7400 0.0030 0.0360 0.0057
log_softmax 1634 50.9330 0.0150 0.0550 0.0312
_backward_copy 1634 84.6190 0.0220 0.0760 0.0518
_backward_Convolution 3268 30225.0762 2.6240 12.5610 9.2488
_mul_scalar 1634 5.8390 0.0020 0.0230 0.0036
_backward_log_softmax 1634 27.9120 0.0090 0.0400 0.0171
mean 5142 98.5280 0.0080 0.0730 0.0192
_backward_mean 1634 17.9630 0.0080 0.1030 0.0110
_backward_FullyConnected 4902 940.3010 0.0280 0.5940 0.1918
WaitForVar 3508 31.2080 0.0060 0.0280 0.0089
broadcast_equal 1874 9.3820 0.0030 0.0110 0.0050
sgd_update 16340 163.3600 0.0020 0.0790 0.0100
_backward_Activation 6536 483.3470 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64132 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5628 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 817 3257.7029 0.3060 73.5870 3.9874
MXNDArrayCreateEx 938 32.5100 0.0310 0.0640 0.0347
MXNDArrayWaitAll 818 17753.2949 5.6130 25.0610 21.7033
MXNDArraySyncCopyToCPU 1756 13829.4482 0.0600 24.1980 7.8755
MXNDArrayGetDType 1756 0.6360 0.0000 0.0010 0.0004
MXNDArrayGetShape 4450 2.0930 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1876 16.5190 0.0070 0.0280 0.0088
MXNDArrayFree 20974 215.7440 0.0000 0.9140 0.0103
MXNet C API Calls 94126 581.4830 487.3580 581.4830 47.0625
MXNet C API Concurrency 188251 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1636 0.7740 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8180 4.2440 0.0000 0.0230 0.0005
MXNDArraySetGradState 8180 4.1800 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1636 0.6980 0.0000 0.0010 0.0004
MXAutogradBackwardEx 818 453.1910 0.5250 0.9450 0.5540
MXNDArrayGetContext 13012 5.7200 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27278 702.0390 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3272 8000.1602 0.7010 3.7500 2.4450
FullyConnected 5628 543.1630 0.0080 0.3430 0.0965
argmax 1876 37.1630 0.0050 0.0480 0.0198
_backward_pick 1636 26.0090 0.0130 0.0320 0.0159
_backward_mul_scalar 1636 7.1580 0.0030 0.0220 0.0044
DeleteVariable 69760 497.3500 0.0010 0.9010 0.0071
Flatten 1876 78.5310 0.0080 0.1120 0.0419
Activation 7504 349.0480 0.0030 1.9440 0.0465
SetValueOp 1636 10.6270 0.0050 0.0180 0.0065
pick 1636 12.8800 0.0060 0.0280 0.0079
Pooling 3752 5225.2070 0.0640 3.1720 1.3926
Convolution 3752 21327.3652 0.2260 13.2050 5.6843
CopyCPU2CPU 1876 10.7500 0.0030 0.0360 0.0057
log_softmax 1636 50.9920 0.0150 0.0550 0.0312
_backward_copy 1636 84.7240 0.0220 0.0760 0.0518
_backward_Convolution 3272 30262.1816 2.6240 12.5610 9.2488
_mul_scalar 1636 5.8450 0.0020 0.0230 0.0036
_backward_log_softmax 1636 27.9440 0.0090 0.0400 0.0171
mean 5148 98.6390 0.0080 0.0730 0.0192
_backward_mean 1636 17.9850 0.0080 0.1030 0.0110
_backward_FullyConnected 4908 941.3690 0.0280 0.5940 0.1918
WaitForVar 3512 31.2420 0.0060 0.0280 0.0089
broadcast_equal 1876 9.3920 0.0030 0.0110 0.0050
sgd_update 16360 163.5630 0.0020 0.0790 0.0100
_backward_Activation 6544 483.9180 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64206 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5634 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 818 3261.6450 0.3060 73.5870 3.9873
MXNDArrayCreateEx 939 32.5460 0.0310 0.0640 0.0347
MXNDArrayWaitAll 819 17775.0391 5.6130 25.0610 21.7033
MXNDArraySyncCopyToCPU 1758 13843.6650 0.0600 24.1980 7.8747
MXNDArrayGetDType 1758 0.6370 0.0000 0.0010 0.0004
MXNDArrayGetShape 4455 2.0950 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1878 16.5370 0.0070 0.0280 0.0088
MXNDArrayFree 20997 215.9810 0.0000 0.9140 0.0103
MXNet C API Calls 94233 581.5900 487.3580 581.5900 47.1160
MXNet C API Concurrency 188465 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1638 0.7750 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8190 4.2500 0.0000 0.0230 0.0005
MXNDArraySetGradState 8190 4.1840 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1638 0.6990 0.0000 0.0010 0.0004
MXAutogradBackwardEx 819 453.7380 0.5250 0.9450 0.5540
MXNDArrayGetContext 13026 5.7220 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27309 702.8650 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3276 8009.8379 0.7010 3.7500 2.4450
FullyConnected 5634 543.6830 0.0080 0.3430 0.0965
argmax 1878 37.2010 0.0050 0.0480 0.0198
_backward_pick 1638 26.0400 0.0130 0.0320 0.0159
_backward_mul_scalar 1638 7.1660 0.0030 0.0220 0.0044
DeleteVariable 69840 497.8750 0.0010 0.9010 0.0071
Flatten 1878 78.6120 0.0080 0.1120 0.0419
Activation 7512 349.4190 0.0030 1.9440 0.0465
SetValueOp 1638 10.6410 0.0050 0.0180 0.0065
pick 1638 12.8950 0.0060 0.0280 0.0079
Pooling 3756 5230.5542 0.0640 3.1720 1.3926
Convolution 3756 21349.5664 0.2260 13.2050 5.6841
CopyCPU2CPU 1878 10.7600 0.0030 0.0360 0.0057
log_softmax 1638 51.0510 0.0150 0.0550 0.0312
_backward_copy 1638 84.8290 0.0220 0.0760 0.0518
_backward_Convolution 3276 30299.0215 2.6240 12.5610 9.2488
_mul_scalar 1638 5.8520 0.0020 0.0230 0.0036
_backward_log_softmax 1638 27.9760 0.0090 0.0400 0.0171
mean 5154 98.7580 0.0080 0.0730 0.0192
_backward_mean 1638 18.0060 0.0080 0.1030 0.0110
_backward_FullyConnected 4914 942.4440 0.0280 0.5940 0.1918
WaitForVar 3516 31.2760 0.0060 0.0280 0.0089
broadcast_equal 1878 9.4010 0.0030 0.0110 0.0050
sgd_update 16380 163.7110 0.0020 0.0790 0.0100
_backward_Activation 6552 484.5240 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64280 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5640 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 819 3265.5649 0.3060 73.5870 3.9873
MXNDArrayCreateEx 940 32.5820 0.0310 0.0640 0.0347
MXNDArrayWaitAll 820 17796.7578 5.6130 25.0610 21.7034
MXNDArraySyncCopyToCPU 1760 13858.2324 0.0600 24.1980 7.8740
MXNDArrayGetDType 1760 0.6380 0.0000 0.0010 0.0004
MXNDArrayGetShape 4460 2.0980 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1880 16.5550 0.0070 0.0280 0.0088
MXNDArrayFree 21020 216.2140 0.0000 0.9140 0.0103
MXNet C API Calls 94340 581.6970 487.3580 581.6970 47.1695
MXNet C API Concurrency 188679 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1640 0.7770 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8200 4.2550 0.0000 0.0230 0.0005
MXNDArraySetGradState 8200 4.1900 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1640 0.7000 0.0000 0.0010 0.0004
MXAutogradBackwardEx 820 454.2900 0.5250 0.9450 0.5540
MXNDArrayGetContext 13040 5.7280 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27340 703.7230 0.0080 0.1450 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3280 8019.6118 0.7010 3.7500 2.4450
FullyConnected 5640 544.4110 0.0080 0.3430 0.0965
argmax 1880 37.2460 0.0050 0.0480 0.0198
_backward_pick 1640 26.0730 0.0130 0.0320 0.0159
_backward_mul_scalar 1640 7.1750 0.0030 0.0220 0.0044
DeleteVariable 69920 498.3990 0.0010 0.9010 0.0071
Flatten 1880 78.6900 0.0080 0.1120 0.0419
Activation 7520 349.8120 0.0030 1.9440 0.0465
SetValueOp 1640 10.6540 0.0050 0.0180 0.0065
pick 1640 12.9120 0.0060 0.0280 0.0079
Pooling 3760 5235.9780 0.0640 3.1720 1.3925
Convolution 3760 21371.7324 0.2260 13.2050 5.6840
CopyCPU2CPU 1880 10.7700 0.0030 0.0360 0.0057
log_softmax 1640 51.1240 0.0150 0.0550 0.0312
_backward_copy 1640 84.9320 0.0220 0.0760 0.0518
_backward_Convolution 3280 30335.7441 2.6240 12.5610 9.2487
_mul_scalar 1640 5.8590 0.0020 0.0230 0.0036
_backward_log_softmax 1640 28.0170 0.0090 0.0400 0.0171
mean 5160 98.8640 0.0080 0.0730 0.0192
_backward_mean 1640 18.0280 0.0080 0.1030 0.0110
_backward_FullyConnected 4920 943.8490 0.0280 0.5940 0.1918
WaitForVar 3520 31.3120 0.0060 0.0280 0.0089
broadcast_equal 1880 9.4100 0.0030 0.0110 0.0050
sgd_update 16400 163.9860 0.0020 0.0790 0.0100
_backward_Activation 6560 485.1160 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64354 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5646 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 820 3269.4929 0.3060 73.5870 3.9872
MXNDArrayCreateEx 941 32.6180 0.0310 0.0640 0.0347
MXNDArrayWaitAll 821 17818.5957 5.6130 25.0610 21.7035
MXNDArraySyncCopyToCPU 1762 13872.0967 0.0600 24.1980 7.8729
MXNDArrayGetDType 1762 0.6390 0.0000 0.0010 0.0004
MXNDArrayGetShape 4465 2.0990 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1882 16.5740 0.0070 0.0280 0.0088
MXNDArrayFree 21043 216.4550 0.0000 0.9140 0.0103
MXNet C API Calls 94447 581.8040 487.3580 581.8040 47.2230
MXNet C API Concurrency 188893 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1642 0.7770 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8210 4.2610 0.0000 0.0230 0.0005
MXNDArraySetGradState 8210 4.1970 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1642 0.7010 0.0000 0.0010 0.0004
MXAutogradBackwardEx 821 454.8590 0.5250 0.9450 0.5540
MXNDArrayGetContext 13054 5.7330 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27371 704.7940 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3284 8029.3198 0.7010 3.7500 2.4450
FullyConnected 5646 544.9590 0.0080 0.3430 0.0965
argmax 1882 37.2840 0.0050 0.0480 0.0198
_backward_pick 1642 26.1040 0.0130 0.0320 0.0159
_backward_mul_scalar 1642 7.1840 0.0030 0.0220 0.0044
DeleteVariable 70000 498.9580 0.0010 0.9010 0.0071
Flatten 1882 78.7680 0.0080 0.1120 0.0419
Activation 7528 350.1800 0.0030 1.9440 0.0465
SetValueOp 1642 10.6660 0.0050 0.0180 0.0065
pick 1642 12.9270 0.0060 0.0280 0.0079
Pooling 3764 5241.3218 0.0640 3.1720 1.3925
Convolution 3764 21393.9180 0.2260 13.2050 5.6838
CopyCPU2CPU 1882 10.7800 0.0030 0.0360 0.0057
log_softmax 1642 51.1830 0.0150 0.0550 0.0312
_backward_copy 1642 85.0370 0.0220 0.0760 0.0518
_backward_Convolution 3284 30372.7676 2.6240 12.5610 9.2487
_mul_scalar 1642 5.8650 0.0020 0.0230 0.0036
_backward_log_softmax 1642 28.0490 0.0090 0.0400 0.0171
mean 5166 98.9780 0.0080 0.0730 0.0192
_backward_mean 1642 18.0500 0.0080 0.1030 0.0110
_backward_FullyConnected 4926 944.9700 0.0280 0.5940 0.1918
WaitForVar 3524 31.3470 0.0060 0.0280 0.0089
broadcast_equal 1882 9.4190 0.0030 0.0110 0.0050
sgd_update 16420 164.1370 0.0020 0.0790 0.0100
_backward_Activation 6568 485.7480 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64428 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5652 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 821 3273.4141 0.3060 73.5870 3.9871
MXNDArrayCreateEx 942 32.6540 0.0310 0.0640 0.0347
MXNDArrayWaitAll 822 17840.4980 5.6130 25.0610 21.7038
MXNDArraySyncCopyToCPU 1764 13886.3672 0.0600 24.1980 7.8721
MXNDArrayGetDType 1764 0.6410 0.0000 0.0010 0.0004
MXNDArrayGetShape 4470 2.1020 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1884 16.5920 0.0070 0.0280 0.0088
MXNDArrayFree 21066 216.6930 0.0000 0.9140 0.0103
MXNet C API Calls 94554 581.9110 487.3580 581.9110 47.2765
MXNet C API Concurrency 189107 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1644 0.7780 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8220 4.2680 0.0000 0.0230 0.0005
MXNDArraySetGradState 8220 4.2000 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1644 0.7010 0.0000 0.0010 0.0004
MXAutogradBackwardEx 822 455.4060 0.5250 0.9450 0.5540
MXNDArrayGetContext 13068 5.7390 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27402 705.6380 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3288 8039.0732 0.7010 3.7500 2.4450
FullyConnected 5652 545.4990 0.0080 0.3430 0.0965
argmax 1884 37.3220 0.0050 0.0480 0.0198
_backward_pick 1644 26.1340 0.0130 0.0320 0.0159
_backward_mul_scalar 1644 7.1930 0.0030 0.0220 0.0044
DeleteVariable 70080 499.5100 0.0010 0.9010 0.0071
Flatten 1884 78.8450 0.0080 0.1120 0.0418
Activation 7536 350.5470 0.0030 1.9440 0.0465
SetValueOp 1644 10.6790 0.0050 0.0180 0.0065
pick 1644 12.9420 0.0060 0.0280 0.0079
Pooling 3768 5246.7388 0.0640 3.1720 1.3924
Convolution 3768 21416.1211 0.2260 13.2050 5.6837
CopyCPU2CPU 1884 10.7900 0.0030 0.0360 0.0057
log_softmax 1644 51.2430 0.0150 0.0550 0.0312
_backward_copy 1644 85.1430 0.0220 0.0760 0.0518
_backward_Convolution 3288 30409.8633 2.6240 12.5610 9.2487
_mul_scalar 1644 5.8720 0.0020 0.0230 0.0036
_backward_log_softmax 1644 28.0800 0.0090 0.0400 0.0171
mean 5172 99.0930 0.0080 0.0730 0.0192
_backward_mean 1644 18.0720 0.0080 0.1030 0.0110
_backward_FullyConnected 4932 946.0430 0.0280 0.5940 0.1918
WaitForVar 3528 31.3830 0.0060 0.0280 0.0089
broadcast_equal 1884 9.4290 0.0030 0.0110 0.0050
sgd_update 16440 164.2910 0.0020 0.0790 0.0100
_backward_Activation 6576 486.3590 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64502 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5658 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 822 3277.3340 0.3060 73.5870 3.9870
MXNDArrayCreateEx 943 32.6890 0.0310 0.0640 0.0347
MXNDArrayWaitAll 823 17862.1426 5.6130 25.0610 21.7037
MXNDArraySyncCopyToCPU 1766 13900.6641 0.0600 24.1980 7.8713
MXNDArrayGetDType 1766 0.6410 0.0000 0.0010 0.0004
MXNDArrayGetShape 4475 2.1060 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1886 16.6100 0.0070 0.0280 0.0088
MXNDArrayFree 21089 216.9270 0.0000 0.9140 0.0103
MXNet C API Calls 94661 582.0180 487.3580 582.0180 47.3300
MXNet C API Concurrency 189321 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1646 0.7800 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8230 4.2730 0.0000 0.0230 0.0005
MXNDArraySetGradState 8230 4.2040 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1646 0.7020 0.0000 0.0010 0.0004
MXAutogradBackwardEx 823 455.9550 0.5250 0.9450 0.5540
MXNDArrayGetContext 13082 5.7430 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27433 706.4530 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3292 8048.6948 0.7010 3.7500 2.4449
FullyConnected 5658 546.0180 0.0080 0.3430 0.0965
argmax 1886 37.3610 0.0050 0.0480 0.0198
_backward_pick 1646 26.1640 0.0130 0.0320 0.0159
_backward_mul_scalar 1646 7.2020 0.0030 0.0220 0.0044
DeleteVariable 70160 500.0360 0.0010 0.9010 0.0071
Flatten 1886 78.9240 0.0080 0.1120 0.0418
Activation 7544 350.9420 0.0030 1.9440 0.0465
SetValueOp 1646 10.6930 0.0050 0.0180 0.0065
pick 1646 12.9580 0.0060 0.0280 0.0079
Pooling 3772 5252.1191 0.0640 3.1720 1.3924
Convolution 3772 21438.2773 0.2260 13.2050 5.6835
CopyCPU2CPU 1886 10.8000 0.0030 0.0360 0.0057
log_softmax 1646 51.3020 0.0150 0.0550 0.0312
_backward_copy 1646 85.2470 0.0220 0.0760 0.0518
_backward_Convolution 3292 30446.5918 2.6240 12.5610 9.2487
_mul_scalar 1646 5.8780 0.0020 0.0230 0.0036
_backward_log_softmax 1646 28.1120 0.0090 0.0400 0.0171
mean 5178 99.2110 0.0080 0.0730 0.0192
_backward_mean 1646 18.0940 0.0080 0.1030 0.0110
_backward_FullyConnected 4938 947.1140 0.0280 0.5940 0.1918
WaitForVar 3532 31.4200 0.0060 0.0280 0.0089
broadcast_equal 1886 9.4390 0.0030 0.0110 0.0050
sgd_update 16460 164.4910 0.0020 0.0790 0.0100
_backward_Activation 6584 486.9640 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64576 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5664 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 823 3281.2610 0.3060 73.5870 3.9870
MXNDArrayCreateEx 944 32.7250 0.0310 0.0640 0.0347
MXNDArrayWaitAll 824 17883.8828 5.6130 25.0610 21.7037
MXNDArraySyncCopyToCPU 1768 13915.2041 0.0600 24.1980 7.8706
MXNDArrayGetDType 1768 0.6410 0.0000 0.0010 0.0004
MXNDArrayGetShape 4480 2.1070 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1888 16.6280 0.0070 0.0280 0.0088
MXNDArrayFree 21112 217.1620 0.0000 0.9140 0.0103
MXNet C API Calls 94768 582.1250 487.3580 582.1250 47.3835
MXNet C API Concurrency 189535 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1648 0.7810 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8240 4.2790 0.0000 0.0230 0.0005
MXNDArraySetGradState 8240 4.2100 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1648 0.7020 0.0000 0.0010 0.0004
MXAutogradBackwardEx 824 456.5070 0.5250 0.9450 0.5540
MXNDArrayGetContext 13096 5.7490 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27464 707.2750 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3296 8058.4028 0.7010 3.7500 2.4449
FullyConnected 5664 546.7800 0.0080 0.3430 0.0965
argmax 1888 37.4060 0.0050 0.0480 0.0198
_backward_pick 1648 26.1970 0.0130 0.0320 0.0159
_backward_mul_scalar 1648 7.2090 0.0030 0.0220 0.0044
DeleteVariable 70240 500.5730 0.0010 0.9010 0.0071
Flatten 1888 79.0020 0.0080 0.1120 0.0418
Activation 7552 351.3160 0.0030 1.9440 0.0465
SetValueOp 1648 10.7060 0.0050 0.0180 0.0065
pick 1648 12.9750 0.0060 0.0280 0.0079
Pooling 3776 5257.5039 0.0640 3.1720 1.3923
Convolution 3776 21460.4727 0.2260 13.2050 5.6834
CopyCPU2CPU 1888 10.8100 0.0030 0.0360 0.0057
log_softmax 1648 51.3750 0.0150 0.0550 0.0312
_backward_copy 1648 85.3490 0.0220 0.0760 0.0518
_backward_Convolution 3296 30483.3730 2.6240 12.5610 9.2486
_mul_scalar 1648 5.8860 0.0020 0.0230 0.0036
_backward_log_softmax 1648 28.1540 0.0090 0.0400 0.0171
mean 5184 99.3170 0.0080 0.0730 0.0192
_backward_mean 1648 18.1160 0.0080 0.1030 0.0110
_backward_FullyConnected 4944 948.5320 0.0280 0.5940 0.1919
WaitForVar 3536 31.4560 0.0060 0.0280 0.0089
broadcast_equal 1888 9.4490 0.0030 0.0110 0.0050
sgd_update 16480 164.7560 0.0020 0.0790 0.0100
_backward_Activation 6592 487.5670 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64650 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5670 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 824 3285.1799 0.3060 73.5870 3.9869
MXNDArrayCreateEx 945 32.7610 0.0310 0.0640 0.0347
MXNDArrayWaitAll 825 17905.7031 5.6130 25.0610 21.7039
MXNDArraySyncCopyToCPU 1770 13929.4365 0.0600 24.1980 7.8697
MXNDArrayGetDType 1770 0.6420 0.0000 0.0010 0.0004
MXNDArrayGetShape 4485 2.1070 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1890 16.6470 0.0070 0.0280 0.0088
MXNDArrayFree 21135 217.3990 0.0000 0.9140 0.0103
MXNet C API Calls 94875 582.2320 487.3580 582.2320 47.4370
MXNet C API Concurrency 189749 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1650 0.7830 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8250 4.2840 0.0000 0.0230 0.0005
MXNDArraySetGradState 8250 4.2160 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1650 0.7030 0.0000 0.0010 0.0004
MXAutogradBackwardEx 825 457.0590 0.5250 0.9450 0.5540
MXNDArrayGetContext 13110 5.7540 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27495 708.1260 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3300 8068.0781 0.7010 3.7500 2.4449
FullyConnected 5670 547.3240 0.0080 0.3430 0.0965
argmax 1890 37.4440 0.0050 0.0480 0.0198
_backward_pick 1650 26.2270 0.0130 0.0320 0.0159
_backward_mul_scalar 1650 7.2180 0.0030 0.0220 0.0044
DeleteVariable 70320 501.1170 0.0010 0.9010 0.0071
Flatten 1890 79.0810 0.0080 0.1120 0.0418
Activation 7560 351.6840 0.0030 1.9440 0.0465
SetValueOp 1650 10.7190 0.0050 0.0180 0.0065
pick 1650 12.9910 0.0060 0.0280 0.0079
Pooling 3780 5262.8862 0.0640 3.1720 1.3923
Convolution 3780 21482.6719 0.2260 13.2050 5.6832
CopyCPU2CPU 1890 10.8190 0.0030 0.0360 0.0057
log_softmax 1650 51.4350 0.0150 0.0550 0.0312
_backward_copy 1650 85.4540 0.0220 0.0760 0.0518
_backward_Convolution 3300 30520.4277 2.6240 12.5610 9.2486
_mul_scalar 1650 5.8920 0.0020 0.0230 0.0036
_backward_log_softmax 1650 28.1860 0.0090 0.0400 0.0171
mean 5190 99.4320 0.0080 0.0730 0.0192
_backward_mean 1650 18.1380 0.0080 0.1030 0.0110
_backward_FullyConnected 4950 949.5940 0.0280 0.5940 0.1918
WaitForVar 3540 31.4910 0.0060 0.0280 0.0089
broadcast_equal 1890 9.4590 0.0030 0.0110 0.0050
sgd_update 16500 164.9480 0.0020 0.0790 0.0100
_backward_Activation 6600 488.1370 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64724 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5676 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 825 3289.0991 0.3060 73.5870 3.9868
MXNDArrayCreateEx 946 32.7970 0.0310 0.0640 0.0347
MXNDArrayWaitAll 826 17927.5684 5.6130 25.0610 21.7041
MXNDArraySyncCopyToCPU 1772 13943.6758 0.0600 24.1980 7.8689
MXNDArrayGetDType 1772 0.6420 0.0000 0.0010 0.0004
MXNDArrayGetShape 4490 2.1100 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1892 16.6650 0.0070 0.0280 0.0088
MXNDArrayFree 21158 217.6430 0.0000 0.9140 0.0103
MXNet C API Calls 94982 582.3390 487.3580 582.3390 47.4905
MXNet C API Concurrency 189963 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1652 0.7840 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8260 4.2870 0.0000 0.0230 0.0005
MXNDArraySetGradState 8260 4.2220 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1652 0.7050 0.0000 0.0010 0.0004
MXAutogradBackwardEx 826 457.6070 0.5250 0.9450 0.5540
MXNDArrayGetContext 13124 5.7620 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27526 708.9520 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3304 8077.7710 0.7010 3.7500 2.4448
FullyConnected 5676 547.8460 0.0080 0.3430 0.0965
argmax 1892 37.4810 0.0050 0.0480 0.0198
_backward_pick 1652 26.2570 0.0130 0.0320 0.0159
_backward_mul_scalar 1652 7.2270 0.0030 0.0220 0.0044
DeleteVariable 70400 501.7190 0.0010 0.9010 0.0071
Flatten 1892 79.1590 0.0080 0.1120 0.0418
Activation 7568 352.0530 0.0030 1.9440 0.0465
SetValueOp 1652 10.7330 0.0050 0.0180 0.0065
pick 1652 13.0060 0.0060 0.0280 0.0079
Pooling 3784 5268.2422 0.0640 3.1720 1.3922
Convolution 3784 21504.9277 0.2260 13.2050 5.6831
CopyCPU2CPU 1892 10.8300 0.0030 0.0360 0.0057
log_softmax 1652 51.4960 0.0150 0.0550 0.0312
_backward_copy 1652 85.5600 0.0220 0.0760 0.0518
_backward_Convolution 3304 30557.5273 2.6240 12.5610 9.2486
_mul_scalar 1652 5.8990 0.0020 0.0230 0.0036
_backward_log_softmax 1652 28.2170 0.0090 0.0400 0.0171
mean 5196 99.5470 0.0080 0.0730 0.0192
_backward_mean 1652 18.1590 0.0080 0.1030 0.0110
_backward_FullyConnected 4956 950.6630 0.0280 0.5940 0.1918
WaitForVar 3544 31.5250 0.0060 0.0280 0.0089
broadcast_equal 1892 9.4690 0.0030 0.0110 0.0050
sgd_update 16520 165.1460 0.0020 0.0790 0.0100
_backward_Activation 6608 488.6840 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64798 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5682 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 826 3293.0271 0.3060 73.5870 3.9867
MXNDArrayCreateEx 947 32.8330 0.0310 0.0640 0.0347
MXNDArrayWaitAll 827 17949.2754 5.6130 25.0610 21.7041
MXNDArraySyncCopyToCPU 1774 13957.8789 0.0600 24.1980 7.8680
MXNDArrayGetDType 1774 0.6430 0.0000 0.0010 0.0004
MXNDArrayGetShape 4495 2.1130 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1894 16.6840 0.0070 0.0280 0.0088
MXNDArrayFree 21181 217.8720 0.0000 0.9140 0.0103
MXNet C API Calls 95089 582.4460 487.3580 582.4460 47.5440
MXNet C API Concurrency 190177 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1654 0.7850 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8270 4.2920 0.0000 0.0230 0.0005
MXNDArraySetGradState 8270 4.2250 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1654 0.7060 0.0000 0.0010 0.0004
MXAutogradBackwardEx 827 458.1600 0.5250 0.9450 0.5540
MXNDArrayGetContext 13138 5.7680 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27557 709.7810 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3308 8087.4600 0.7010 3.7500 2.4448
FullyConnected 5682 548.3930 0.0080 0.3430 0.0965
argmax 1894 37.5180 0.0050 0.0480 0.0198
_backward_pick 1654 26.2870 0.0130 0.0320 0.0159
_backward_mul_scalar 1654 7.2360 0.0030 0.0220 0.0044
DeleteVariable 70480 502.2710 0.0010 0.9010 0.0071
Flatten 1894 79.2370 0.0080 0.1120 0.0418
Activation 7576 352.4220 0.0030 1.9440 0.0465
SetValueOp 1654 10.7460 0.0050 0.0180 0.0065
pick 1654 13.0220 0.0060 0.0280 0.0079
Pooling 3788 5273.6099 0.0640 3.1720 1.3922
Convolution 3788 21527.0918 0.2260 13.2050 5.6830
CopyCPU2CPU 1894 10.8400 0.0030 0.0360 0.0057
log_softmax 1654 51.5560 0.0150 0.0550 0.0312
_backward_copy 1654 85.6650 0.0220 0.0760 0.0518
_backward_Convolution 3308 30594.3203 2.6240 12.5610 9.2486
_mul_scalar 1654 5.9060 0.0020 0.0230 0.0036
_backward_log_softmax 1654 28.2500 0.0090 0.0400 0.0171
mean 5202 99.6600 0.0080 0.0730 0.0192
_backward_mean 1654 18.1810 0.0080 0.1030 0.0110
_backward_FullyConnected 4962 951.7340 0.0280 0.5940 0.1918
WaitForVar 3548 31.5620 0.0060 0.0280 0.0089
broadcast_equal 1894 9.4780 0.0030 0.0110 0.0050
sgd_update 16540 165.3460 0.0020 0.0790 0.0100
_backward_Activation 6616 489.2660 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64872 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5688 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 827 3296.9570 0.3060 73.5870 3.9866
MXNDArrayCreateEx 948 32.8690 0.0310 0.0640 0.0347
MXNDArrayWaitAll 828 17971.0254 5.6130 25.0610 21.7041
MXNDArraySyncCopyToCPU 1776 13972.4297 0.0600 24.1980 7.8674
MXNDArrayGetDType 1776 0.6430 0.0000 0.0010 0.0004
MXNDArrayGetShape 4500 2.1130 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1896 16.7010 0.0070 0.0280 0.0088
MXNDArrayFree 21204 218.1060 0.0000 0.9140 0.0103
MXNet C API Calls 95196 582.5530 487.3580 582.5530 47.5975
MXNet C API Concurrency 190391 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1656 0.7850 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8280 4.2940 0.0000 0.0230 0.0005
MXNDArraySetGradState 8280 4.2310 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1656 0.7060 0.0000 0.0010 0.0004
MXAutogradBackwardEx 828 458.7090 0.5250 0.9450 0.5540
MXNDArrayGetContext 13152 5.7750 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27588 710.6250 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3312 8097.2109 0.7010 3.7500 2.4448
FullyConnected 5688 549.1190 0.0080 0.3430 0.0965
argmax 1896 37.5620 0.0050 0.0480 0.0198
_backward_pick 1656 26.3190 0.0130 0.0320 0.0159
_backward_mul_scalar 1656 7.2450 0.0030 0.0220 0.0044
DeleteVariable 70560 502.8160 0.0010 0.9010 0.0071
Flatten 1896 79.3160 0.0080 0.1120 0.0418
Activation 7584 352.7970 0.0030 1.9440 0.0465
SetValueOp 1656 10.7580 0.0050 0.0180 0.0065
pick 1656 13.0390 0.0060 0.0280 0.0079
Pooling 3792 5278.9390 0.0640 3.1720 1.3921
Convolution 3792 21549.2793 0.2260 13.2050 5.6828
CopyCPU2CPU 1896 10.8500 0.0030 0.0360 0.0057
log_softmax 1656 51.6290 0.0150 0.0550 0.0312
_backward_copy 1656 85.7670 0.0220 0.0760 0.0518
_backward_Convolution 3312 30631.0938 2.6240 12.5610 9.2485
_mul_scalar 1656 5.9120 0.0020 0.0230 0.0036
_backward_log_softmax 1656 28.2930 0.0090 0.0400 0.0171
mean 5208 99.7680 0.0080 0.0730 0.0192
_backward_mean 1656 18.2030 0.0080 0.1030 0.0110
_backward_FullyConnected 4968 953.2060 0.0280 0.5940 0.1919
WaitForVar 3552 31.5960 0.0060 0.0280 0.0089
broadcast_equal 1896 9.4880 0.0030 0.0110 0.0050
sgd_update 16560 165.6180 0.0020 0.0790 0.0100
_backward_Activation 6624 489.8740 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 64946 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5694 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 828 3300.8799 0.3060 73.5870 3.9866
MXNDArrayCreateEx 949 32.9040 0.0310 0.0640 0.0347
MXNDArrayWaitAll 829 17992.8320 5.6130 25.0610 21.7043
MXNDArraySyncCopyToCPU 1778 13986.7217 0.0600 24.1980 7.8665
MXNDArrayGetDType 1778 0.6430 0.0000 0.0010 0.0004
MXNDArrayGetShape 4505 2.1140 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1898 16.7190 0.0070 0.0280 0.0088
MXNDArrayFree 21227 218.3440 0.0000 0.9140 0.0103
MXNet C API Calls 95303 582.6600 487.3580 582.6600 47.6510
MXNet C API Concurrency 190605 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1658 0.7870 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8290 4.2950 0.0000 0.0230 0.0005
MXNDArraySetGradState 8290 4.2360 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1658 0.7070 0.0000 0.0010 0.0004
MXAutogradBackwardEx 829 459.2610 0.5250 0.9450 0.5540
MXNDArrayGetContext 13166 5.7850 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27619 711.4590 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3316 8107.0381 0.7010 3.7500 2.4448
FullyConnected 5694 549.6630 0.0080 0.3430 0.0965
argmax 1898 37.6010 0.0050 0.0480 0.0198
_backward_pick 1658 26.3500 0.0130 0.0320 0.0159
_backward_mul_scalar 1658 7.2540 0.0030 0.0220 0.0044
DeleteVariable 70640 503.3770 0.0010 0.9010 0.0071
Flatten 1898 79.3960 0.0080 0.1120 0.0418
Activation 7592 353.1680 0.0030 1.9440 0.0465
SetValueOp 1658 10.7710 0.0050 0.0180 0.0065
pick 1658 13.0560 0.0060 0.0280 0.0079
Pooling 3796 5284.3730 0.0640 3.1720 1.3921
Convolution 3796 21571.4238 0.2260 13.2050 5.6827
CopyCPU2CPU 1898 10.8600 0.0030 0.0360 0.0057
log_softmax 1658 51.6890 0.0150 0.0550 0.0312
_backward_copy 1658 85.8740 0.0220 0.0760 0.0518
_backward_Convolution 3316 30668.0508 2.6240 12.5610 9.2485
_mul_scalar 1658 5.9190 0.0020 0.0230 0.0036
_backward_log_softmax 1658 28.3240 0.0090 0.0400 0.0171
mean 5214 99.8780 0.0080 0.0730 0.0192
_backward_mean 1658 18.2250 0.0080 0.1030 0.0110
_backward_FullyConnected 4974 954.2780 0.0280 0.5940 0.1919
WaitForVar 3556 31.6320 0.0060 0.0280 0.0089
broadcast_equal 1898 9.4970 0.0030 0.0110 0.0050
sgd_update 16580 165.8180 0.0020 0.0790 0.0100
_backward_Activation 6632 490.4600 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65020 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5700 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 829 3304.7949 0.3060 73.5870 3.9865
MXNDArrayCreateEx 950 32.9400 0.0310 0.0640 0.0347
MXNDArrayWaitAll 830 18014.6953 5.6130 25.0610 21.7045
MXNDArraySyncCopyToCPU 1780 14001.0361 0.0600 24.1980 7.8658
MXNDArrayGetDType 1780 0.6440 0.0000 0.0010 0.0004
MXNDArrayGetShape 4510 2.1150 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1900 16.7380 0.0070 0.0280 0.0088
MXNDArrayFree 21250 218.5760 0.0000 0.9140 0.0103
MXNet C API Calls 95410 582.7670 487.3580 582.7670 47.7045
MXNet C API Concurrency 190819 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1660 0.7880 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8300 4.2970 0.0000 0.0230 0.0005
MXNDArraySetGradState 8300 4.2420 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1660 0.7090 0.0000 0.0010 0.0004
MXAutogradBackwardEx 830 459.8080 0.5250 0.9450 0.5540
MXNDArrayGetContext 13180 5.7930 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27650 712.2830 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3320 8116.8018 0.7010 3.7500 2.4448
FullyConnected 5700 550.2070 0.0080 0.3430 0.0965
argmax 1900 37.6390 0.0050 0.0480 0.0198
_backward_pick 1660 26.3780 0.0130 0.0320 0.0159
_backward_mul_scalar 1660 7.2630 0.0030 0.0220 0.0044
DeleteVariable 70720 503.9150 0.0010 0.9010 0.0071
Flatten 1900 79.4730 0.0080 0.1120 0.0418
Activation 7600 353.5630 0.0030 1.9440 0.0465
SetValueOp 1660 10.7840 0.0050 0.0180 0.0065
pick 1660 13.0710 0.0060 0.0280 0.0079
Pooling 3800 5289.7500 0.0640 3.1720 1.3920
Convolution 3800 21593.6621 0.2260 13.2050 5.6825
CopyCPU2CPU 1900 10.8700 0.0030 0.0360 0.0057
log_softmax 1660 51.7480 0.0150 0.0550 0.0312
_backward_copy 1660 85.9800 0.0220 0.0760 0.0518
_backward_Convolution 3320 30705.0918 2.6240 12.5610 9.2485
_mul_scalar 1660 5.9260 0.0020 0.0230 0.0036
_backward_log_softmax 1660 28.3560 0.0090 0.0400 0.0171
mean 5220 99.9940 0.0080 0.0730 0.0192
_backward_mean 1660 18.2460 0.0080 0.1030 0.0110
_backward_FullyConnected 4980 955.3460 0.0280 0.5940 0.1918
WaitForVar 3560 31.6680 0.0060 0.0280 0.0089
broadcast_equal 1900 9.5080 0.0030 0.0110 0.0050
sgd_update 16600 166.0190 0.0020 0.0790 0.0100
_backward_Activation 6640 491.0540 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65094 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5706 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 830 3308.7241 0.3060 73.5870 3.9864
MXNDArrayCreateEx 951 32.9760 0.0310 0.0640 0.0347
MXNDArrayWaitAll 831 18036.4492 5.6130 25.0610 21.7045
MXNDArraySyncCopyToCPU 1782 14015.3086 0.0600 24.1980 7.8649
MXNDArrayGetDType 1782 0.6460 0.0000 0.0010 0.0004
MXNDArrayGetShape 4515 2.1160 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1902 16.7550 0.0070 0.0280 0.0088
MXNDArrayFree 21273 218.8130 0.0000 0.9140 0.0103
MXNet C API Calls 95517 582.8740 487.3580 582.8740 47.7580
MXNet C API Concurrency 191033 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1662 0.7900 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8310 4.3020 0.0000 0.0230 0.0005
MXNDArraySetGradState 8310 4.2470 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1662 0.7090 0.0000 0.0010 0.0004
MXAutogradBackwardEx 831 460.3810 0.5250 0.9450 0.5540
MXNDArrayGetContext 13194 5.7960 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27681 713.0940 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3324 8126.5591 0.7010 3.7500 2.4448
FullyConnected 5706 550.7530 0.0080 0.3430 0.0965
argmax 1902 37.6760 0.0050 0.0480 0.0198
_backward_pick 1662 26.4090 0.0130 0.0320 0.0159
_backward_mul_scalar 1662 7.2710 0.0030 0.0220 0.0044
DeleteVariable 70800 504.4390 0.0010 0.9010 0.0071
Flatten 1902 79.5550 0.0080 0.1120 0.0418
Activation 7608 353.9310 0.0030 1.9440 0.0465
SetValueOp 1662 10.7970 0.0050 0.0180 0.0065
pick 1662 13.0860 0.0060 0.0280 0.0079
Pooling 3804 5295.1162 0.0640 3.1720 1.3920
Convolution 3804 21615.8691 0.2260 13.2050 5.6824
CopyCPU2CPU 1902 10.8800 0.0030 0.0360 0.0057
log_softmax 1662 51.8070 0.0150 0.0550 0.0312
_backward_copy 1662 86.0860 0.0220 0.0760 0.0518
_backward_Convolution 3324 30741.9375 2.6240 12.5610 9.2485
_mul_scalar 1662 5.9330 0.0020 0.0230 0.0036
_backward_log_softmax 1662 28.3880 0.0090 0.0400 0.0171
mean 5226 100.1060 0.0080 0.0730 0.0192
_backward_mean 1662 18.2680 0.0080 0.1030 0.0110
_backward_FullyConnected 4986 956.4170 0.0280 0.5940 0.1918
WaitForVar 3564 31.7040 0.0060 0.0280 0.0089
broadcast_equal 1902 9.5170 0.0030 0.0110 0.0050
sgd_update 16620 166.2160 0.0020 0.0790 0.0100
_backward_Activation 6648 491.6210 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65168 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5712 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 831 3312.6421 0.3060 73.5870 3.9863
MXNDArrayCreateEx 952 33.0120 0.0310 0.0640 0.0347
MXNDArrayWaitAll 832 18058.1367 5.6130 25.0610 21.7045
MXNDArraySyncCopyToCPU 1784 14029.9258 0.0600 24.1980 7.8643
MXNDArrayGetDType 1784 0.6460 0.0000 0.0010 0.0004
MXNDArrayGetShape 4520 2.1190 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1904 16.7740 0.0070 0.0280 0.0088
MXNDArrayFree 21296 219.0470 0.0000 0.9140 0.0103
MXNet C API Calls 95624 582.9810 487.3580 582.9810 47.8115
MXNet C API Concurrency 191247 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1664 0.7910 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8320 4.3070 0.0000 0.0230 0.0005
MXNDArraySetGradState 8320 4.2520 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1664 0.7090 0.0000 0.0010 0.0004
MXAutogradBackwardEx 832 460.9290 0.5250 0.9450 0.5540
MXNDArrayGetContext 13208 5.7990 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27712 713.9470 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3328 8136.3901 0.7010 3.7500 2.4448
FullyConnected 5712 551.4780 0.0080 0.3430 0.0965
argmax 1904 37.7220 0.0050 0.0480 0.0198
_backward_pick 1664 26.4410 0.0130 0.0320 0.0159
_backward_mul_scalar 1664 7.2800 0.0030 0.0220 0.0044
DeleteVariable 70880 504.9860 0.0010 0.9010 0.0071
Flatten 1904 79.6320 0.0080 0.1120 0.0418
Activation 7616 354.3110 0.0030 1.9440 0.0465
SetValueOp 1664 10.8090 0.0050 0.0180 0.0065
pick 1664 13.1030 0.0060 0.0280 0.0079
Pooling 3808 5300.5122 0.0640 3.1720 1.3919
Convolution 3808 21638.1270 0.2260 13.2050 5.6823
CopyCPU2CPU 1904 10.8890 0.0030 0.0360 0.0057
log_softmax 1664 51.8780 0.0150 0.0550 0.0312
_backward_copy 1664 86.1870 0.0220 0.0760 0.0518
_backward_Convolution 3328 30778.6074 2.6240 12.5610 9.2484
_mul_scalar 1664 5.9390 0.0020 0.0230 0.0036
_backward_log_softmax 1664 28.4300 0.0090 0.0400 0.0171
mean 5232 100.2100 0.0080 0.0730 0.0192
_backward_mean 1664 18.2900 0.0080 0.1030 0.0110
_backward_FullyConnected 4992 957.8170 0.0280 0.5940 0.1919
WaitForVar 3568 31.7390 0.0060 0.0280 0.0089
broadcast_equal 1904 9.5270 0.0030 0.0110 0.0050
sgd_update 16640 166.4800 0.0020 0.0790 0.0100
_backward_Activation 6656 492.2190 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65242 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5718 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 832 3316.5669 0.3060 73.5870 3.9863
MXNDArrayCreateEx 953 33.0480 0.0310 0.0640 0.0347
MXNDArrayWaitAll 833 18079.9883 5.6130 25.0610 21.7047
MXNDArraySyncCopyToCPU 1786 14044.1865 0.0600 24.1980 7.8635
MXNDArrayGetDType 1786 0.6480 0.0000 0.0010 0.0004
MXNDArrayGetShape 4525 2.1200 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1906 16.7930 0.0070 0.0280 0.0088
MXNDArrayFree 21319 219.2830 0.0000 0.9140 0.0103
MXNet C API Calls 95731 583.0880 487.3580 583.0880 47.8650
MXNet C API Concurrency 191461 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1666 0.7920 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8330 4.3140 0.0000 0.0230 0.0005
MXNDArraySetGradState 8330 4.2550 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1666 0.7100 0.0000 0.0010 0.0004
MXAutogradBackwardEx 833 461.4800 0.5250 0.9450 0.5540
MXNDArrayGetContext 13222 5.8090 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27743 714.7950 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3332 8146.1138 0.7010 3.7500 2.4448
FullyConnected 5718 552.0240 0.0080 0.3430 0.0965
argmax 1906 37.7600 0.0050 0.0480 0.0198
_backward_pick 1666 26.4720 0.0130 0.0320 0.0159
_backward_mul_scalar 1666 7.2890 0.0030 0.0220 0.0044
DeleteVariable 70960 505.5750 0.0010 0.9010 0.0071
Flatten 1906 79.7120 0.0080 0.1120 0.0418
Activation 7624 354.6790 0.0030 1.9440 0.0465
SetValueOp 1666 10.8220 0.0050 0.0180 0.0065
pick 1666 13.1180 0.0060 0.0280 0.0079
Pooling 3812 5305.8511 0.0640 3.1720 1.3919
Convolution 3812 21660.3379 0.2260 13.2050 5.6821
CopyCPU2CPU 1906 10.8990 0.0030 0.0360 0.0057
log_softmax 1666 51.9370 0.0150 0.0550 0.0312
_backward_copy 1666 86.2940 0.0220 0.0760 0.0518
_backward_Convolution 3332 30815.6660 2.6240 12.5610 9.2484
_mul_scalar 1666 5.9470 0.0020 0.0230 0.0036
_backward_log_softmax 1666 28.4630 0.0090 0.0400 0.0171
mean 5238 100.3280 0.0080 0.0730 0.0192
_backward_mean 1666 18.3120 0.0080 0.1030 0.0110
_backward_FullyConnected 4998 958.8830 0.0280 0.5940 0.1919
WaitForVar 3572 31.7740 0.0060 0.0280 0.0089
broadcast_equal 1906 9.5360 0.0030 0.0110 0.0050
sgd_update 16660 166.6790 0.0020 0.0790 0.0100
_backward_Activation 6664 492.7770 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65316 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5724 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 833 3320.4961 0.3060 73.5870 3.9862
MXNDArrayCreateEx 954 33.0840 0.0310 0.0640 0.0347
MXNDArrayWaitAll 834 18101.8789 5.6130 25.0610 21.7049
MXNDArraySyncCopyToCPU 1788 14058.4346 0.0600 24.1980 7.8627
MXNDArrayGetDType 1788 0.6500 0.0000 0.0010 0.0004
MXNDArrayGetShape 4530 2.1210 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1908 16.8100 0.0070 0.0280 0.0088
MXNDArrayFree 21342 219.5260 0.0000 0.9140 0.0103
MXNet C API Calls 95838 583.1950 487.3580 583.1950 47.9185
MXNet C API Concurrency 191675 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1668 0.7930 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8340 4.3170 0.0000 0.0230 0.0005
MXNDArraySetGradState 8340 4.2610 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1668 0.7120 0.0000 0.0010 0.0004
MXAutogradBackwardEx 834 462.0280 0.5250 0.9450 0.5540
MXNDArrayGetContext 13236 5.8130 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27774 715.6540 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3336 8155.8872 0.7010 3.7500 2.4448
FullyConnected 5724 552.5470 0.0080 0.3430 0.0965
argmax 1908 37.7990 0.0050 0.0480 0.0198
_backward_pick 1668 26.5010 0.0130 0.0320 0.0159
_backward_mul_scalar 1668 7.2960 0.0030 0.0220 0.0044
DeleteVariable 71040 506.1390 0.0010 0.9010 0.0071
Flatten 1908 79.7920 0.0080 0.1120 0.0418
Activation 7632 355.0480 0.0030 1.9440 0.0465
SetValueOp 1668 10.8360 0.0050 0.0180 0.0065
pick 1668 13.1330 0.0060 0.0280 0.0079
Pooling 3816 5311.2090 0.0640 3.1720 1.3918
Convolution 3816 21682.6191 0.2260 13.2050 5.6820
CopyCPU2CPU 1908 10.9090 0.0030 0.0360 0.0057
log_softmax 1668 51.9970 0.0150 0.0550 0.0312
_backward_copy 1668 86.4010 0.0220 0.0760 0.0518
_backward_Convolution 3336 30852.7422 2.6240 12.5610 9.2484
_mul_scalar 1668 5.9550 0.0020 0.0230 0.0036
_backward_log_softmax 1668 28.4940 0.0090 0.0400 0.0171
mean 5244 100.4450 0.0080 0.0730 0.0192
_backward_mean 1668 18.3330 0.0080 0.1030 0.0110
_backward_FullyConnected 5004 959.9500 0.0280 0.5940 0.1918
WaitForVar 3576 31.8100 0.0060 0.0280 0.0089
broadcast_equal 1908 9.5460 0.0030 0.0110 0.0050
sgd_update 16680 166.8270 0.0020 0.0790 0.0100
_backward_Activation 6672 493.4130 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65390 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5730 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 834 3324.4280 0.3060 73.5870 3.9861
MXNDArrayCreateEx 955 33.1190 0.0310 0.0640 0.0347
MXNDArrayWaitAll 835 18123.6367 5.6130 25.0610 21.7050
MXNDArraySyncCopyToCPU 1790 14072.7129 0.0600 24.1980 7.8619
MXNDArrayGetDType 1790 0.6500 0.0000 0.0010 0.0004
MXNDArrayGetShape 4535 2.1230 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1910 16.8280 0.0070 0.0280 0.0088
MXNDArrayFree 21365 219.7640 0.0000 0.9140 0.0103
MXNet C API Calls 95945 583.3020 487.3580 583.3020 47.9720
MXNet C API Concurrency 191889 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1670 0.7930 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8350 4.3210 0.0000 0.0230 0.0005
MXNDArraySetGradState 8350 4.2660 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1670 0.7120 0.0000 0.0010 0.0004
MXAutogradBackwardEx 835 462.5750 0.5250 0.9450 0.5540
MXNDArrayGetContext 13250 5.8210 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27805 716.4790 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3340 8165.7231 0.7010 3.7500 2.4448
FullyConnected 5730 553.0680 0.0080 0.3430 0.0965
argmax 1910 37.8380 0.0050 0.0480 0.0198
_backward_pick 1670 26.5340 0.0130 0.0320 0.0159
_backward_mul_scalar 1670 7.3060 0.0030 0.0220 0.0044
DeleteVariable 71120 506.6850 0.0010 0.9010 0.0071
Flatten 1910 79.8700 0.0080 0.1120 0.0418
Activation 7640 355.4160 0.0030 1.9440 0.0465
SetValueOp 1670 10.8500 0.0050 0.0180 0.0065
pick 1670 13.1480 0.0060 0.0280 0.0079
Pooling 3820 5316.5659 0.0640 3.1720 1.3918
Convolution 3820 21704.8281 0.2260 13.2050 5.6819
CopyCPU2CPU 1910 10.9190 0.0030 0.0360 0.0057
log_softmax 1670 52.0570 0.0150 0.0550 0.0312
_backward_copy 1670 86.5050 0.0220 0.0760 0.0518
_backward_Convolution 3340 30889.5273 2.6240 12.5610 9.2484
_mul_scalar 1670 5.9630 0.0020 0.0230 0.0036
_backward_log_softmax 1670 28.5260 0.0090 0.0400 0.0171
mean 5250 100.5620 0.0080 0.0730 0.0192
_backward_mean 1670 18.3550 0.0080 0.1030 0.0110
_backward_FullyConnected 5010 961.0680 0.0280 0.5940 0.1918
WaitForVar 3580 31.8460 0.0060 0.0280 0.0089
broadcast_equal 1910 9.5550 0.0030 0.0110 0.0050
sgd_update 16700 167.0330 0.0020 0.0790 0.0100
_backward_Activation 6680 493.9960 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65464 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5736 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 835 3328.3789 0.3060 73.5870 3.9861
MXNDArrayCreateEx 956 33.1560 0.0310 0.0640 0.0347
MXNDArrayWaitAll 836 18145.3770 5.6130 25.0610 21.7050
MXNDArraySyncCopyToCPU 1792 14087.3223 0.0600 24.1980 7.8612
MXNDArrayGetDType 1792 0.6510 0.0000 0.0010 0.0004
MXNDArrayGetShape 4540 2.1270 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1912 16.8460 0.0070 0.0280 0.0088
MXNDArrayFree 21388 220.0000 0.0000 0.9140 0.0103
MXNet C API Calls 96052 583.4090 487.3580 583.4090 48.0255
MXNet C API Concurrency 192103 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1672 0.7940 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8360 4.3290 0.0000 0.0230 0.0005
MXNDArraySetGradState 8360 4.2710 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1672 0.7130 0.0000 0.0010 0.0004
MXAutogradBackwardEx 836 463.1460 0.5250 0.9450 0.5540
MXNDArrayGetContext 13264 5.8240 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27836 717.2970 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3344 8175.6108 0.7010 3.7500 2.4449
FullyConnected 5736 553.7990 0.0080 0.3430 0.0965
argmax 1912 37.8830 0.0050 0.0480 0.0198
_backward_pick 1672 26.5680 0.0130 0.0320 0.0159
_backward_mul_scalar 1672 7.3150 0.0030 0.0220 0.0044
DeleteVariable 71200 507.2380 0.0010 0.9010 0.0071
Flatten 1912 79.9500 0.0080 0.1120 0.0418
Activation 7648 355.7910 0.0030 1.9440 0.0465
SetValueOp 1672 10.8630 0.0050 0.0180 0.0065
pick 1672 13.1650 0.0060 0.0280 0.0079
Pooling 3824 5322.0049 0.0640 3.1720 1.3917
Convolution 3824 21727.0020 0.2260 13.2050 5.6817
CopyCPU2CPU 1912 10.9280 0.0030 0.0360 0.0057
log_softmax 1672 52.1290 0.0150 0.0550 0.0312
_backward_copy 1672 86.6070 0.0220 0.0760 0.0518
_backward_Convolution 3344 30926.2246 2.6240 12.5610 9.2483
_mul_scalar 1672 5.9700 0.0020 0.0230 0.0036
_backward_log_softmax 1672 28.5690 0.0090 0.0400 0.0171
mean 5256 100.6730 0.0080 0.0730 0.0192
_backward_mean 1672 18.3760 0.0080 0.1030 0.0110
_backward_FullyConnected 5016 962.4830 0.0280 0.5940 0.1919
WaitForVar 3584 31.8810 0.0060 0.0280 0.0089
broadcast_equal 1912 9.5640 0.0030 0.0110 0.0050
sgd_update 16720 167.3030 0.0020 0.0790 0.0100
_backward_Activation 6688 494.5900 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65538 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5742 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 836 3332.3049 0.3060 73.5870 3.9860
MXNDArrayCreateEx 957 33.1890 0.0310 0.0640 0.0347
MXNDArrayWaitAll 837 18167.2109 5.6130 25.0610 21.7051
MXNDArraySyncCopyToCPU 1794 14101.6855 0.0600 24.1980 7.8605
MXNDArrayGetDType 1794 0.6510 0.0000 0.0010 0.0004
MXNDArrayGetShape 4545 2.1300 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1914 16.8650 0.0070 0.0280 0.0088
MXNDArrayFree 21411 220.2320 0.0000 0.9140 0.0103
MXNet C API Calls 96159 583.5160 487.3580 583.5160 48.0790
MXNet C API Concurrency 192317 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1674 0.7940 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8370 4.3340 0.0000 0.0230 0.0005
MXNDArraySetGradState 8370 4.2760 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1674 0.7130 0.0000 0.0010 0.0004
MXAutogradBackwardEx 837 463.6930 0.5250 0.9450 0.5540
MXNDArrayGetContext 13278 5.8310 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27867 718.0940 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3348 8185.3950 0.7010 3.7500 2.4449
FullyConnected 5742 554.3210 0.0080 0.3430 0.0965
argmax 1914 37.9210 0.0050 0.0480 0.0198
_backward_pick 1674 26.5980 0.0130 0.0320 0.0159
_backward_mul_scalar 1674 7.3240 0.0030 0.0220 0.0044
DeleteVariable 71280 507.7540 0.0010 0.9010 0.0071
Flatten 1914 80.0300 0.0080 0.1120 0.0418
Activation 7656 356.1610 0.0030 1.9440 0.0465
SetValueOp 1674 10.8770 0.0050 0.0180 0.0065
pick 1674 13.1800 0.0060 0.0280 0.0079
Pooling 3828 5327.3359 0.0640 3.1720 1.3917
Convolution 3828 21749.1797 0.2260 13.2050 5.6816
CopyCPU2CPU 1914 10.9370 0.0030 0.0360 0.0057
log_softmax 1674 52.1890 0.0150 0.0550 0.0312
_backward_copy 1674 86.7130 0.0220 0.0760 0.0518
_backward_Convolution 3348 30963.2383 2.6240 12.5610 9.2483
_mul_scalar 1674 5.9770 0.0020 0.0230 0.0036
_backward_log_softmax 1674 28.6000 0.0090 0.0400 0.0171
mean 5262 100.7910 0.0080 0.0730 0.0192
_backward_mean 1674 18.3980 0.0080 0.1030 0.0110
_backward_FullyConnected 5022 963.5590 0.0280 0.5940 0.1919
WaitForVar 3588 31.9170 0.0060 0.0280 0.0089
broadcast_equal 1914 9.5740 0.0030 0.0110 0.0050
sgd_update 16740 167.5250 0.0020 0.0790 0.0100
_backward_Activation 6696 495.1720 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65612 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5748 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 837 3336.2161 0.3060 73.5870 3.9859
MXNDArrayCreateEx 958 33.2220 0.0310 0.0640 0.0347
MXNDArrayWaitAll 838 18189.0820 5.6130 25.0610 21.7053
MXNDArraySyncCopyToCPU 1796 14116.1055 0.0600 24.1980 7.8597
MXNDArrayGetDType 1796 0.6530 0.0000 0.0010 0.0004
MXNDArrayGetShape 4550 2.1330 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1916 16.8810 0.0070 0.0280 0.0088
MXNDArrayFree 21434 220.4680 0.0000 0.9140 0.0103
MXNet C API Calls 96266 583.6230 487.3580 583.6230 48.1325
MXNet C API Concurrency 192531 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1676 0.7950 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8380 4.3380 0.0000 0.0230 0.0005
MXNDArraySetGradState 8380 4.2820 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1676 0.7130 0.0000 0.0010 0.0004
MXAutogradBackwardEx 838 464.2560 0.5250 0.9450 0.5540
MXNDArrayGetContext 13292 5.8380 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27898 718.8690 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3352 8195.1182 0.7010 3.7500 2.4448
FullyConnected 5748 554.8450 0.0080 0.3430 0.0965
argmax 1916 37.9600 0.0050 0.0480 0.0198
_backward_pick 1676 26.6290 0.0130 0.0320 0.0159
_backward_mul_scalar 1676 7.3330 0.0030 0.0220 0.0044
DeleteVariable 71360 508.2980 0.0010 0.9010 0.0071
Flatten 1916 80.1080 0.0080 0.1120 0.0418
Activation 7664 356.5320 0.0030 1.9440 0.0465
SetValueOp 1676 10.8910 0.0050 0.0180 0.0065
pick 1676 13.1950 0.0060 0.0280 0.0079
Pooling 3832 5332.7319 0.0640 3.1720 1.3916
Convolution 3832 21771.4258 0.2260 13.2050 5.6815
CopyCPU2CPU 1916 10.9470 0.0030 0.0360 0.0057
log_softmax 1676 52.2480 0.0150 0.0550 0.0312
_backward_copy 1676 86.8180 0.0220 0.0760 0.0518
_backward_Convolution 3352 31000.2715 2.6240 12.5610 9.2483
_mul_scalar 1676 5.9840 0.0020 0.0230 0.0036
_backward_log_softmax 1676 28.6320 0.0090 0.0400 0.0171
mean 5268 100.9130 0.0080 0.0730 0.0192
_backward_mean 1676 18.4190 0.0080 0.1030 0.0110
_backward_FullyConnected 5028 964.6340 0.0280 0.5940 0.1919
WaitForVar 3592 31.9520 0.0060 0.0280 0.0089
broadcast_equal 1916 9.5830 0.0030 0.0110 0.0050
sgd_update 16760 167.7290 0.0020 0.0790 0.0100
_backward_Activation 6704 495.7420 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65686 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5754 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 838 3337.9810 0.3060 73.5870 3.9833
MXNDArrayCreateEx 959 33.2560 0.0310 0.0640 0.0347
MXNDArrayWaitAll 839 18210.7812 5.6130 25.0610 21.7053
MXNDArraySyncCopyToCPU 1798 14130.4619 0.0600 24.1980 7.8590
MXNDArrayGetDType 1798 0.6540 0.0000 0.0010 0.0004
MXNDArrayGetShape 4555 2.1350 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1918 16.8990 0.0070 0.0280 0.0088
MXNDArrayFree 21457 220.7010 0.0000 0.9140 0.0103
MXNet C API Calls 96373 583.7300 487.3580 583.7300 48.1860
MXNet C API Concurrency 192745 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1678 0.7950 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8390 4.3420 0.0000 0.0230 0.0005
MXNDArraySetGradState 8390 4.2880 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1678 0.7130 0.0000 0.0010 0.0004
MXAutogradBackwardEx 839 464.8270 0.5250 0.9450 0.5540
MXNDArrayGetContext 13306 5.8440 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27929 719.6570 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3356 8204.7285 0.7010 3.7500 2.4448
FullyConnected 5754 555.3950 0.0080 0.3430 0.0965
argmax 1918 37.9990 0.0050 0.0480 0.0198
_backward_pick 1678 26.6600 0.0130 0.0320 0.0159
_backward_mul_scalar 1678 7.3410 0.0030 0.0220 0.0044
DeleteVariable 71440 508.8170 0.0010 0.9010 0.0071
Flatten 1918 80.1890 0.0080 0.1120 0.0418
Activation 7672 356.9020 0.0030 1.9440 0.0465
SetValueOp 1678 10.9030 0.0050 0.0180 0.0065
pick 1678 13.2090 0.0060 0.0280 0.0079
Pooling 3836 5338.0889 0.0640 3.1720 1.3916
Convolution 3836 21793.6582 0.2260 13.2050 5.6814
CopyCPU2CPU 1918 10.9570 0.0030 0.0360 0.0057
log_softmax 1678 52.3080 0.0150 0.0550 0.0312
_backward_copy 1678 86.9220 0.0220 0.0760 0.0518
_backward_Convolution 3356 31037.0742 2.6240 12.5610 9.2482
_mul_scalar 1678 5.9910 0.0020 0.0230 0.0036
_backward_log_softmax 1678 28.6640 0.0090 0.0400 0.0171
mean 5274 101.0300 0.0080 0.0730 0.0192
_backward_mean 1678 18.4410 0.0080 0.1030 0.0110
_backward_FullyConnected 5034 965.7090 0.0280 0.5940 0.1918
WaitForVar 3596 31.9880 0.0060 0.0280 0.0089
broadcast_equal 1918 9.5930 0.0030 0.0110 0.0050
sgd_update 16780 167.8780 0.0020 0.0790 0.0100
_backward_Activation 6712 496.3850 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65760 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5760 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 839 3341.9089 0.3060 73.5870 3.9832
MXNDArrayCreateEx 960 33.2900 0.0310 0.0640 0.0347
MXNDArrayWaitAll 840 18232.4805 5.6130 25.0610 21.7053
MXNDArraySyncCopyToCPU 1800 14145.2803 0.0600 24.1980 7.8585
MXNDArrayGetDType 1800 0.6540 0.0000 0.0010 0.0004
MXNDArrayGetShape 4560 2.1370 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1920 16.9160 0.0070 0.0280 0.0088
MXNDArrayFree 21480 220.9320 0.0000 0.9140 0.0103
MXNet C API Calls 96480 583.8370 487.3580 583.8370 48.2395
MXNet C API Concurrency 192959 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1680 0.7970 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8400 4.3460 0.0000 0.0230 0.0005
MXNDArraySetGradState 8400 4.2960 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1680 0.7140 0.0000 0.0010 0.0004
MXAutogradBackwardEx 840 465.3680 0.5250 0.9450 0.5540
MXNDArrayGetContext 13320 5.8510 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27960 720.4390 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3360 8214.5439 0.7010 3.7500 2.4448
FullyConnected 5760 556.1220 0.0080 0.3430 0.0965
argmax 1920 38.0430 0.0050 0.0480 0.0198
_backward_pick 1680 26.6950 0.0130 0.0320 0.0159
_backward_mul_scalar 1680 7.3490 0.0030 0.0220 0.0044
DeleteVariable 71520 509.3680 0.0010 0.9010 0.0071
Flatten 1920 80.2700 0.0080 0.1120 0.0418
Activation 7680 357.2870 0.0030 1.9440 0.0465
SetValueOp 1680 10.9150 0.0050 0.0180 0.0065
pick 1680 13.2260 0.0060 0.0280 0.0079
Pooling 3840 5343.4922 0.0640 3.1720 1.3915
Convolution 3840 21815.8867 0.2260 13.2050 5.6812
CopyCPU2CPU 1920 10.9670 0.0030 0.0360 0.0057
log_softmax 1680 52.3800 0.0150 0.0550 0.0312
_backward_copy 1680 87.0230 0.0220 0.0760 0.0518
_backward_Convolution 3360 31073.8223 2.6240 12.5610 9.2482
_mul_scalar 1680 5.9980 0.0020 0.0230 0.0036
_backward_log_softmax 1680 28.7060 0.0090 0.0400 0.0171
mean 5280 101.1340 0.0080 0.0730 0.0192
_backward_mean 1680 18.4640 0.0080 0.1030 0.0110
_backward_FullyConnected 5040 967.1190 0.0280 0.5940 0.1919
WaitForVar 3600 32.0240 0.0060 0.0280 0.0089
broadcast_equal 1920 9.6040 0.0030 0.0110 0.0050
sgd_update 16800 168.1520 0.0020 0.0790 0.0100
_backward_Activation 6720 497.0060 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65834 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5766 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 840 3345.8159 0.3060 73.5870 3.9831
MXNDArrayCreateEx 961 33.3240 0.0310 0.0640 0.0347
MXNDArrayWaitAll 841 18254.3203 5.6130 25.0610 21.7055
MXNDArraySyncCopyToCPU 1802 14159.7070 0.0600 24.1980 7.8578
MXNDArrayGetDType 1802 0.6560 0.0000 0.0010 0.0004
MXNDArrayGetShape 4565 2.1400 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1922 16.9330 0.0070 0.0280 0.0088
MXNDArrayFree 21503 221.1630 0.0000 0.9140 0.0103
MXNet C API Calls 96587 583.9440 487.3580 583.9440 48.2930
MXNet C API Concurrency 193173 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1682 0.7990 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8410 4.3500 0.0000 0.0230 0.0005
MXNDArraySetGradState 8410 4.3030 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1682 0.7150 0.0000 0.0010 0.0004
MXAutogradBackwardEx 841 465.9040 0.5250 0.9450 0.5540
MXNDArrayGetContext 13334 5.8550 0.0000 0.0190 0.0004
MXImperativeInvokeEx 27991 721.2240 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3364 8224.3008 0.7010 3.7500 2.4448
FullyConnected 5766 556.6450 0.0080 0.3430 0.0965
argmax 1922 38.0800 0.0050 0.0480 0.0198
_backward_pick 1682 26.7260 0.0130 0.0320 0.0159
_backward_mul_scalar 1682 7.3580 0.0030 0.0220 0.0044
DeleteVariable 71600 509.9510 0.0010 0.9010 0.0071
Flatten 1922 80.3520 0.0080 0.1120 0.0418
Activation 7688 357.6580 0.0030 1.9440 0.0465
SetValueOp 1682 10.9290 0.0050 0.0180 0.0065
pick 1682 13.2410 0.0060 0.0280 0.0079
Pooling 3844 5348.8618 0.0640 3.1720 1.3915
Convolution 3844 21838.0742 0.2260 13.2050 5.6811
CopyCPU2CPU 1922 10.9770 0.0030 0.0360 0.0057
log_softmax 1682 52.4390 0.0150 0.0550 0.0312
_backward_copy 1682 87.1300 0.0220 0.0760 0.0518
_backward_Convolution 3364 31110.8320 2.6240 12.5610 9.2482
_mul_scalar 1682 6.0050 0.0020 0.0230 0.0036
_backward_log_softmax 1682 28.7380 0.0090 0.0400 0.0171
mean 5286 101.2520 0.0080 0.0730 0.0192
_backward_mean 1682 18.4860 0.0080 0.1030 0.0110
_backward_FullyConnected 5046 968.1870 0.0280 0.5940 0.1919
WaitForVar 3604 32.0590 0.0060 0.0280 0.0089
broadcast_equal 1922 9.6140 0.0030 0.0110 0.0050
sgd_update 16820 168.2980 0.0020 0.0790 0.0100
_backward_Activation 6728 497.6100 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65908 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5772 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 841 3349.7141 0.3060 73.5870 3.9830
MXNDArrayCreateEx 962 33.3580 0.0310 0.0640 0.0347
MXNDArrayWaitAll 842 18276.1641 5.6130 25.0610 21.7057
MXNDArraySyncCopyToCPU 1804 14173.9766 0.0600 24.1980 7.8570
MXNDArrayGetDType 1804 0.6580 0.0000 0.0010 0.0004
MXNDArrayGetShape 4570 2.1430 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1924 16.9500 0.0070 0.0280 0.0088
MXNDArrayFree 21526 221.3900 0.0000 0.9140 0.0103
MXNet C API Calls 96694 584.0510 487.3580 584.0510 48.3465
MXNet C API Concurrency 193387 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1684 0.8000 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8420 4.3570 0.0000 0.0230 0.0005
MXNDArraySetGradState 8420 4.3110 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1684 0.7150 0.0000 0.0010 0.0004
MXAutogradBackwardEx 842 466.4340 0.5250 0.9450 0.5540
MXNDArrayGetContext 13348 5.8600 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28022 722.0000 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3368 8234.0781 0.7010 3.7500 2.4448
FullyConnected 5772 557.1630 0.0080 0.3430 0.0965
argmax 1924 38.1180 0.0050 0.0480 0.0198
_backward_pick 1684 26.7570 0.0130 0.0320 0.0159
_backward_mul_scalar 1684 7.3660 0.0030 0.0220 0.0044
DeleteVariable 71680 510.4630 0.0010 0.9010 0.0071
Flatten 1924 80.4300 0.0080 0.1120 0.0418
Activation 7696 358.0260 0.0030 1.9440 0.0465
SetValueOp 1684 10.9430 0.0050 0.0180 0.0065
pick 1684 13.2570 0.0060 0.0280 0.0079
Pooling 3848 5354.2388 0.0640 3.1720 1.3914
Convolution 3848 21860.3574 0.2260 13.2050 5.6810
CopyCPU2CPU 1924 10.9870 0.0030 0.0360 0.0057
log_softmax 1684 52.4990 0.0150 0.0550 0.0312
_backward_copy 1684 87.2370 0.0220 0.0760 0.0518
_backward_Convolution 3368 31147.8516 2.6240 12.5610 9.2482
_mul_scalar 1684 6.0110 0.0020 0.0230 0.0036
_backward_log_softmax 1684 28.7710 0.0090 0.0400 0.0171
mean 5292 101.3690 0.0080 0.0730 0.0192
_backward_mean 1684 18.5080 0.0080 0.1030 0.0110
_backward_FullyConnected 5052 969.2530 0.0280 0.5940 0.1919
WaitForVar 3608 32.0950 0.0060 0.0280 0.0089
broadcast_equal 1924 9.6240 0.0030 0.0110 0.0050
sgd_update 16840 168.4490 0.0020 0.0790 0.0100
_backward_Activation 6736 498.2490 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 65982 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5778 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 842 3353.6069 0.3060 73.5870 3.9829
MXNDArrayCreateEx 963 33.3970 0.0310 0.0640 0.0347
MXNDArrayWaitAll 843 18297.8555 5.6130 25.0610 21.7056
MXNDArraySyncCopyToCPU 1806 14188.1230 0.0600 24.1980 7.8561
MXNDArrayGetDType 1806 0.6580 0.0000 0.0010 0.0004
MXNDArrayGetShape 4575 2.1470 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1926 16.9670 0.0070 0.0280 0.0088
MXNDArrayFree 21549 221.6290 0.0000 0.9140 0.0103
MXNet C API Calls 96801 584.1580 487.3580 584.1580 48.4000
MXNet C API Concurrency 193601 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1686 0.8010 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8430 4.3630 0.0000 0.0230 0.0005
MXNDArraySetGradState 8430 4.3150 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1686 0.7160 0.0000 0.0010 0.0004
MXAutogradBackwardEx 843 466.9910 0.5250 0.9450 0.5540
MXNDArrayGetContext 13362 5.8660 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28053 722.8080 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3372 8243.7861 0.7010 3.7500 2.4448
FullyConnected 5778 557.7130 0.0080 0.3430 0.0965
argmax 1926 38.1570 0.0050 0.0480 0.0198
_backward_pick 1686 26.7880 0.0130 0.0320 0.0159
_backward_mul_scalar 1686 7.3740 0.0030 0.0220 0.0044
DeleteVariable 71760 511.0010 0.0010 0.9010 0.0071
Flatten 1926 80.5080 0.0080 0.1120 0.0418
Activation 7704 358.3960 0.0030 1.9440 0.0465
SetValueOp 1686 10.9560 0.0050 0.0180 0.0065
pick 1686 13.2730 0.0060 0.0280 0.0079
Pooling 3852 5359.5571 0.0640 3.1720 1.3914
Convolution 3852 21882.5703 0.2260 13.2050 5.6808
CopyCPU2CPU 1926 10.9980 0.0030 0.0360 0.0057
log_softmax 1686 52.5580 0.0150 0.0550 0.0312
_backward_copy 1686 87.3400 0.0220 0.0760 0.0518
_backward_Convolution 3372 31184.5957 2.6240 12.5610 9.2481
_mul_scalar 1686 6.0190 0.0020 0.0230 0.0036
_backward_log_softmax 1686 28.8030 0.0090 0.0400 0.0171
mean 5298 101.4860 0.0080 0.0730 0.0192
_backward_mean 1686 18.5290 0.0080 0.1030 0.0110
_backward_FullyConnected 5058 970.3740 0.0280 0.5940 0.1918
WaitForVar 3612 32.1320 0.0060 0.0280 0.0089
broadcast_equal 1926 9.6330 0.0030 0.0110 0.0050
sgd_update 16860 168.6490 0.0020 0.0790 0.0100
_backward_Activation 6744 498.8180 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66056 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5784 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 843 3357.5330 0.3060 73.5870 3.9828
MXNDArrayCreateEx 964 33.4310 0.0310 0.0640 0.0347
MXNDArrayWaitAll 844 18319.6191 5.6130 25.0610 21.7057
MXNDArraySyncCopyToCPU 1808 14202.8848 0.0600 24.1980 7.8556
MXNDArrayGetDType 1808 0.6590 0.0000 0.0010 0.0004
MXNDArrayGetShape 4580 2.1490 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1928 16.9850 0.0070 0.0280 0.0088
MXNDArrayFree 21572 221.8560 0.0000 0.9140 0.0103
MXNet C API Calls 96908 584.2650 487.3580 584.2650 48.4535
MXNet C API Concurrency 193815 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1688 0.8030 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8440 4.3680 0.0000 0.0230 0.0005
MXNDArraySetGradState 8440 4.3190 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1688 0.7170 0.0000 0.0010 0.0004
MXAutogradBackwardEx 844 467.5600 0.5250 0.9450 0.5540
MXNDArrayGetContext 13376 5.8720 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28084 723.5900 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3376 8253.6426 0.7010 3.7500 2.4448
FullyConnected 5784 558.4460 0.0080 0.3430 0.0966
argmax 1928 38.2010 0.0050 0.0480 0.0198
_backward_pick 1688 26.8190 0.0130 0.0320 0.0159
_backward_mul_scalar 1688 7.3830 0.0030 0.0220 0.0044
DeleteVariable 71840 511.5260 0.0010 0.9010 0.0071
Flatten 1928 80.5880 0.0080 0.1120 0.0418
Activation 7712 358.7690 0.0030 1.9440 0.0465
SetValueOp 1688 10.9680 0.0050 0.0180 0.0065
pick 1688 13.2900 0.0060 0.0280 0.0079
Pooling 3856 5364.9429 0.0640 3.1720 1.3913
Convolution 3856 21904.7832 0.2260 13.2050 5.6807
CopyCPU2CPU 1928 11.0080 0.0030 0.0360 0.0057
log_softmax 1688 52.6310 0.0150 0.0550 0.0312
_backward_copy 1688 87.4430 0.0220 0.0760 0.0518
_backward_Convolution 3376 31221.3672 2.6240 12.5610 9.2480
_mul_scalar 1688 6.0260 0.0020 0.0230 0.0036
_backward_log_softmax 1688 28.8450 0.0090 0.0400 0.0171
mean 5304 101.5940 0.0080 0.0730 0.0192
_backward_mean 1688 18.5510 0.0080 0.1030 0.0110
_backward_FullyConnected 5064 971.7960 0.0280 0.5940 0.1919
WaitForVar 3616 32.1680 0.0060 0.0280 0.0089
broadcast_equal 1928 9.6430 0.0030 0.0110 0.0050
sgd_update 16880 168.9230 0.0020 0.0790 0.0100
_backward_Activation 6752 499.4080 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66130 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5790 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 844 3361.4341 0.3060 73.5870 3.9827
MXNDArrayCreateEx 965 33.4640 0.0310 0.0640 0.0347
MXNDArrayWaitAll 845 18341.4199 5.6130 25.0610 21.7058
MXNDArraySyncCopyToCPU 1810 14217.3350 0.0600 24.1980 7.8549
MXNDArrayGetDType 1810 0.6590 0.0000 0.0010 0.0004
MXNDArrayGetShape 4585 2.1490 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1930 17.0030 0.0070 0.0280 0.0088
MXNDArrayFree 21595 222.0980 0.0000 0.9140 0.0103
MXNet C API Calls 97015 584.3720 487.3580 584.3720 48.5070
MXNet C API Concurrency 194029 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1690 0.8040 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8450 4.3730 0.0000 0.0230 0.0005
MXNDArraySetGradState 8450 4.3230 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1690 0.7170 0.0000 0.0010 0.0004
MXAutogradBackwardEx 845 468.0950 0.5250 0.9450 0.5540
MXNDArrayGetContext 13390 5.8800 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28115 724.3580 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3380 8263.3604 0.7010 3.7500 2.4448
FullyConnected 5790 558.9790 0.0080 0.3430 0.0965
argmax 1930 38.2400 0.0050 0.0480 0.0198
_backward_pick 1690 26.8500 0.0130 0.0320 0.0159
_backward_mul_scalar 1690 7.3910 0.0030 0.0220 0.0044
DeleteVariable 71920 512.0810 0.0010 0.9010 0.0071
Flatten 1930 80.6670 0.0080 0.1120 0.0418
Activation 7720 359.1740 0.0030 1.9440 0.0465
SetValueOp 1690 10.9810 0.0050 0.0180 0.0065
pick 1690 13.3070 0.0060 0.0280 0.0079
Pooling 3860 5370.2510 0.0640 3.1720 1.3913
Convolution 3860 21926.9395 0.2260 13.2050 5.6806
CopyCPU2CPU 1930 11.0170 0.0030 0.0360 0.0057
log_softmax 1690 52.6910 0.0150 0.0550 0.0312
_backward_copy 1690 87.5480 0.0220 0.0760 0.0518
_backward_Convolution 3380 31258.3652 2.6240 12.5610 9.2480
_mul_scalar 1690 6.0330 0.0020 0.0230 0.0036
_backward_log_softmax 1690 28.8770 0.0090 0.0400 0.0171
mean 5310 101.7120 0.0080 0.0730 0.0192
_backward_mean 1690 18.5730 0.0080 0.1030 0.0110
_backward_FullyConnected 5070 972.8670 0.0280 0.5940 0.1919
WaitForVar 3620 32.2030 0.0060 0.0280 0.0089
broadcast_equal 1930 9.6530 0.0030 0.0110 0.0050
sgd_update 16900 169.1270 0.0020 0.0790 0.0100
_backward_Activation 6760 499.9940 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66204 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5796 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 845 3365.3560 0.3060 73.5870 3.9827
MXNDArrayCreateEx 966 33.4970 0.0310 0.0640 0.0347
MXNDArrayWaitAll 846 18363.3125 5.6130 25.0610 21.7060
MXNDArraySyncCopyToCPU 1812 14231.8252 0.0600 24.1980 7.8542
MXNDArrayGetDType 1812 0.6590 0.0000 0.0010 0.0004
MXNDArrayGetShape 4590 2.1500 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1932 17.0200 0.0070 0.0280 0.0088
MXNDArrayFree 21618 222.3250 0.0000 0.9140 0.0103
MXNet C API Calls 97122 584.4790 487.3580 584.4790 48.5605
MXNet C API Concurrency 194243 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1692 0.8060 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8460 4.3780 0.0000 0.0230 0.0005
MXNDArraySetGradState 8460 4.3270 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1692 0.7180 0.0000 0.0010 0.0004
MXAutogradBackwardEx 846 468.6230 0.5250 0.9450 0.5539
MXNDArrayGetContext 13404 5.8870 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28146 725.1280 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3384 8273.1201 0.7010 3.7500 2.4448
FullyConnected 5796 559.5600 0.0080 0.3430 0.0965
argmax 1932 38.2780 0.0050 0.0480 0.0198
_backward_pick 1692 26.8810 0.0130 0.0320 0.0159
_backward_mul_scalar 1692 7.4000 0.0030 0.0220 0.0044
DeleteVariable 72000 512.6150 0.0010 0.9010 0.0071
Flatten 1932 80.7510 0.0080 0.1120 0.0418
Activation 7728 359.5440 0.0030 1.9440 0.0465
SetValueOp 1692 10.9960 0.0050 0.0180 0.0065
pick 1692 13.3220 0.0060 0.0280 0.0079
Pooling 3864 5375.6152 0.0640 3.1720 1.3912
Convolution 3864 21949.2070 0.2260 13.2050 5.6804
CopyCPU2CPU 1932 11.0270 0.0030 0.0360 0.0057
log_softmax 1692 52.7500 0.0150 0.0550 0.0312
_backward_copy 1692 87.6530 0.0220 0.0760 0.0518
_backward_Convolution 3384 31295.4336 2.6240 12.5610 9.2481
_mul_scalar 1692 6.0400 0.0020 0.0230 0.0036
_backward_log_softmax 1692 28.9080 0.0090 0.0400 0.0171
mean 5316 101.8460 0.0080 0.0730 0.0192
_backward_mean 1692 18.5940 0.0080 0.1030 0.0110
_backward_FullyConnected 5076 973.9370 0.0280 0.5940 0.1919
WaitForVar 3624 32.2390 0.0060 0.0280 0.0089
broadcast_equal 1932 9.6630 0.0030 0.0110 0.0050
sgd_update 16920 169.3350 0.0020 0.0790 0.0100
_backward_Activation 6768 500.5900 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66278 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5802 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 846 3369.2529 0.3060 73.5870 3.9826
MXNDArrayCreateEx 967 33.5310 0.0310 0.0640 0.0347
MXNDArrayWaitAll 847 18385.0059 5.6130 25.0610 21.7060
MXNDArraySyncCopyToCPU 1814 14246.2842 0.0600 24.1980 7.8535
MXNDArrayGetDType 1814 0.6610 0.0000 0.0010 0.0004
MXNDArrayGetShape 4595 2.1530 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1934 17.0370 0.0070 0.0280 0.0088
MXNDArrayFree 21641 222.5550 0.0000 0.9140 0.0103
MXNet C API Calls 97229 584.5860 487.3580 584.5860 48.6140
MXNet C API Concurrency 194457 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1694 0.8080 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8470 4.3830 0.0000 0.0230 0.0005
MXNDArraySetGradState 8470 4.3310 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1694 0.7190 0.0000 0.0010 0.0004
MXAutogradBackwardEx 847 469.1530 0.5250 0.9450 0.5539
MXNDArrayGetContext 13418 5.8910 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28177 725.8950 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3388 8282.8711 0.7010 3.7500 2.4448
FullyConnected 5802 560.0830 0.0080 0.3430 0.0965
argmax 1934 38.3150 0.0050 0.0480 0.0198
_backward_pick 1694 26.9130 0.0130 0.0320 0.0159
_backward_mul_scalar 1694 7.4090 0.0030 0.0220 0.0044
DeleteVariable 72080 513.1430 0.0010 0.9010 0.0071
Flatten 1934 80.8300 0.0080 0.1120 0.0418
Activation 7736 359.9120 0.0030 1.9440 0.0465
SetValueOp 1694 11.0090 0.0050 0.0180 0.0065
pick 1694 13.3370 0.0060 0.0280 0.0079
Pooling 3868 5381.0249 0.0640 3.1720 1.3912
Convolution 3868 21971.3926 0.2260 13.2050 5.6803
CopyCPU2CPU 1934 11.0360 0.0030 0.0360 0.0057
log_softmax 1694 52.8090 0.0150 0.0550 0.0312
_backward_copy 1694 87.7580 0.0220 0.0760 0.0518
_backward_Convolution 3388 31332.1426 2.6240 12.5610 9.2480
_mul_scalar 1694 6.0470 0.0020 0.0230 0.0036
_backward_log_softmax 1694 28.9400 0.0090 0.0400 0.0171
mean 5322 101.9610 0.0080 0.0730 0.0192
_backward_mean 1694 18.6150 0.0080 0.1030 0.0110
_backward_FullyConnected 5082 975.0090 0.0280 0.5940 0.1919
WaitForVar 3628 32.2760 0.0060 0.0280 0.0089
broadcast_equal 1934 9.6740 0.0030 0.0110 0.0050
sgd_update 16940 169.5410 0.0020 0.0790 0.0100
_backward_Activation 6776 501.1910 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66352 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5808 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 847 3373.1489 0.3060 73.5870 3.9825
MXNDArrayCreateEx 968 33.5650 0.0310 0.0640 0.0347
MXNDArrayWaitAll 848 18406.7656 5.6130 25.0610 21.7061
MXNDArraySyncCopyToCPU 1816 14261.0654 0.0600 24.1980 7.8530
MXNDArrayGetDType 1816 0.6610 0.0000 0.0010 0.0004
MXNDArrayGetShape 4600 2.1550 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1936 17.0540 0.0070 0.0280 0.0088
MXNDArrayFree 21664 222.7800 0.0000 0.9140 0.0103
MXNet C API Calls 97336 584.6930 487.3580 584.6930 48.6675
MXNet C API Concurrency 194671 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1696 0.8080 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8480 4.3900 0.0000 0.0230 0.0005
MXNDArraySetGradState 8480 4.3340 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1696 0.7200 0.0000 0.0010 0.0004
MXAutogradBackwardEx 848 469.6820 0.5250 0.9450 0.5539
MXNDArrayGetContext 13432 5.8960 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28208 726.6590 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3392 8292.6553 0.7010 3.7500 2.4448
FullyConnected 5808 560.8130 0.0080 0.3430 0.0966
argmax 1936 38.3590 0.0050 0.0480 0.0198
_backward_pick 1696 26.9460 0.0130 0.0320 0.0159
_backward_mul_scalar 1696 7.4180 0.0030 0.0220 0.0044
DeleteVariable 72160 513.6510 0.0010 0.9010 0.0071
Flatten 1936 80.9090 0.0080 0.1120 0.0418
Activation 7744 360.2880 0.0030 1.9440 0.0465
SetValueOp 1696 11.0220 0.0050 0.0180 0.0065
pick 1696 13.3540 0.0060 0.0280 0.0079
Pooling 3872 5386.3501 0.0640 3.1720 1.3911
Convolution 3872 21993.6172 0.2260 13.2050 5.6802
CopyCPU2CPU 1936 11.0460 0.0030 0.0360 0.0057
log_softmax 1696 52.8820 0.0150 0.0550 0.0312
_backward_copy 1696 87.8600 0.0220 0.0760 0.0518
_backward_Convolution 3392 31368.9336 2.6240 12.5610 9.2479
_mul_scalar 1696 6.0540 0.0020 0.0230 0.0036
_backward_log_softmax 1696 28.9820 0.0090 0.0400 0.0171
mean 5328 102.0740 0.0080 0.0730 0.0192
_backward_mean 1696 18.6360 0.0080 0.1030 0.0110
_backward_FullyConnected 5088 976.4760 0.0280 0.5940 0.1919
WaitForVar 3632 32.3080 0.0060 0.0280 0.0089
broadcast_equal 1936 9.6830 0.0030 0.0110 0.0050
sgd_update 16960 169.8140 0.0020 0.0790 0.0100
_backward_Activation 6784 501.7750 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66426 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5814 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 848 3377.0449 0.3060 73.5870 3.9824
MXNDArrayCreateEx 969 33.5980 0.0310 0.0640 0.0347
MXNDArrayWaitAll 849 18428.6543 5.6130 25.0610 21.7063
MXNDArraySyncCopyToCPU 1818 14275.3652 0.0600 24.1980 7.8522
MXNDArrayGetDType 1818 0.6620 0.0000 0.0010 0.0004
MXNDArrayGetShape 4605 2.1560 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1938 17.0710 0.0070 0.0280 0.0088
MXNDArrayFree 21687 223.0120 0.0000 0.9140 0.0103
MXNet C API Calls 97443 584.8000 487.3580 584.8000 48.7210
MXNet C API Concurrency 194885 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1698 0.8100 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8490 4.3950 0.0000 0.0230 0.0005
MXNDArraySetGradState 8490 4.3390 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1698 0.7210 0.0000 0.0010 0.0004
MXAutogradBackwardEx 849 470.2350 0.5250 0.9450 0.5539
MXNDArrayGetContext 13446 5.9020 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28239 727.4540 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3396 8302.5039 0.7010 3.7500 2.4448
FullyConnected 5814 561.3600 0.0080 0.3430 0.0966
argmax 1938 38.3980 0.0050 0.0480 0.0198
_backward_pick 1698 26.9780 0.0130 0.0320 0.0159
_backward_mul_scalar 1698 7.4270 0.0030 0.0220 0.0044
DeleteVariable 72240 514.2120 0.0010 0.9010 0.0071
Flatten 1938 80.9890 0.0080 0.1120 0.0418
Activation 7752 360.6570 0.0030 1.9440 0.0465
SetValueOp 1698 11.0350 0.0050 0.0180 0.0065
pick 1698 13.3680 0.0060 0.0280 0.0079
Pooling 3876 5391.7310 0.0640 3.1720 1.3911
Convolution 3876 22015.7734 0.2260 13.2050 5.6800
CopyCPU2CPU 1938 11.0550 0.0030 0.0360 0.0057
log_softmax 1698 52.9410 0.0150 0.0550 0.0312
_backward_copy 1698 87.9660 0.0220 0.0760 0.0518
_backward_Convolution 3396 31405.9785 2.6240 12.5610 9.2479
_mul_scalar 1698 6.0610 0.0020 0.0230 0.0036
_backward_log_softmax 1698 29.0150 0.0090 0.0400 0.0171
mean 5334 102.1950 0.0080 0.0730 0.0192
_backward_mean 1698 18.6570 0.0080 0.1030 0.0110
_backward_FullyConnected 5094 977.5500 0.0280 0.5940 0.1919
WaitForVar 3636 32.3440 0.0060 0.0280 0.0089
broadcast_equal 1938 9.6930 0.0030 0.0110 0.0050
sgd_update 16980 169.9650 0.0020 0.0790 0.0100
_backward_Activation 6792 502.4040 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66500 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5820 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 849 3380.9470 0.3060 73.5870 3.9823
MXNDArrayCreateEx 970 33.6310 0.0310 0.0640 0.0347
MXNDArrayWaitAll 850 18450.5391 5.6130 25.0610 21.7065
MXNDArraySyncCopyToCPU 1820 14289.8398 0.0600 24.1980 7.8516
MXNDArrayGetDType 1820 0.6640 0.0000 0.0010 0.0004
MXNDArrayGetShape 4610 2.1600 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1940 17.0890 0.0070 0.0280 0.0088
MXNDArrayFree 21710 223.2360 0.0000 0.9140 0.0103
MXNet C API Calls 97550 584.9070 487.3580 584.9070 48.7745
MXNet C API Concurrency 195099 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1700 0.8110 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8500 4.4010 0.0000 0.0230 0.0005
MXNDArraySetGradState 8500 4.3450 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1700 0.7220 0.0000 0.0010 0.0004
MXAutogradBackwardEx 850 470.7690 0.5250 0.9450 0.5538
MXNDArrayGetContext 13460 5.9100 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28270 728.2260 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3400 8312.1904 0.7010 3.7500 2.4448
FullyConnected 5820 561.8840 0.0080 0.3430 0.0965
argmax 1940 38.4730 0.0050 0.0480 0.0198
_backward_pick 1700 27.0090 0.0130 0.0320 0.0159
_backward_mul_scalar 1700 7.4350 0.0030 0.0220 0.0044
DeleteVariable 72320 514.7320 0.0010 0.9010 0.0071
Flatten 1940 81.0650 0.0080 0.1120 0.0418
Activation 7760 361.0260 0.0030 1.9440 0.0465
SetValueOp 1700 11.0470 0.0050 0.0180 0.0065
pick 1700 13.3840 0.0060 0.0280 0.0079
Pooling 3880 5397.0688 0.0640 3.1720 1.3910
Convolution 3880 22038.0664 0.2260 13.2050 5.6799
CopyCPU2CPU 1940 11.0650 0.0030 0.0360 0.0057
log_softmax 1700 53.0010 0.0150 0.0550 0.0312
_backward_copy 1700 88.0700 0.0220 0.0760 0.0518
_backward_Convolution 3400 31443.1055 2.6240 12.5610 9.2480
_mul_scalar 1700 6.0670 0.0020 0.0230 0.0036
_backward_log_softmax 1700 29.0470 0.0090 0.0400 0.0171
mean 5340 102.3130 0.0080 0.0730 0.0192
_backward_mean 1700 18.6790 0.0080 0.1030 0.0110
_backward_FullyConnected 5100 978.6150 0.0280 0.5940 0.1919
WaitForVar 3640 32.3810 0.0060 0.0280 0.0089
broadcast_equal 1940 9.7030 0.0030 0.0110 0.0050
sgd_update 17000 170.1670 0.0020 0.0790 0.0100
_backward_Activation 6800 502.9740 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66574 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5826 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 850 3384.8250 0.3060 73.5870 3.9821
MXNDArrayCreateEx 971 33.6640 0.0310 0.0640 0.0347
MXNDArrayWaitAll 851 18472.3027 5.6130 25.0610 21.7066
MXNDArraySyncCopyToCPU 1822 14304.2754 0.0600 24.1980 7.8509
MXNDArrayGetDType 1822 0.6650 0.0000 0.0010 0.0004
MXNDArrayGetShape 4615 2.1600 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1942 17.1070 0.0070 0.0280 0.0088
MXNDArrayFree 21733 223.4620 0.0000 0.9140 0.0103
MXNet C API Calls 97657 585.0140 487.3580 585.0140 48.8280
MXNet C API Concurrency 195313 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1702 0.8110 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8510 4.4080 0.0000 0.0230 0.0005
MXNDArraySetGradState 8510 4.3480 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1702 0.7240 0.0000 0.0010 0.0004
MXAutogradBackwardEx 851 471.3180 0.5250 0.9450 0.5538
MXNDArrayGetContext 13474 5.9170 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28301 728.9890 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3404 8321.9590 0.7010 3.7500 2.4448
FullyConnected 5826 562.4050 0.0080 0.3430 0.0965
argmax 1942 38.5110 0.0050 0.0480 0.0198
_backward_pick 1702 27.0390 0.0130 0.0320 0.0159
_backward_mul_scalar 1702 7.4440 0.0030 0.0220 0.0044
DeleteVariable 72400 515.2460 0.0010 0.9010 0.0071
Flatten 1942 81.1450 0.0080 0.1120 0.0418
Activation 7768 361.3960 0.0030 1.9440 0.0465
SetValueOp 1702 11.0600 0.0050 0.0180 0.0065
pick 1702 13.4000 0.0060 0.0280 0.0079
Pooling 3884 5402.4580 0.0640 3.1720 1.3910
Convolution 3884 22060.2422 0.2260 13.2050 5.6798
CopyCPU2CPU 1942 11.0750 0.0030 0.0360 0.0057
log_softmax 1702 53.0600 0.0150 0.0550 0.0312
_backward_copy 1702 88.1760 0.0220 0.0760 0.0518
_backward_Convolution 3404 31479.9297 2.6240 12.5610 9.2479
_mul_scalar 1702 6.0740 0.0020 0.0230 0.0036
_backward_log_softmax 1702 29.0790 0.0090 0.0400 0.0171
mean 5346 102.4320 0.0080 0.0730 0.0192
_backward_mean 1702 18.7000 0.0080 0.1030 0.0110
_backward_FullyConnected 5106 979.6880 0.0280 0.5940 0.1919
WaitForVar 3644 32.4160 0.0060 0.0280 0.0089
broadcast_equal 1942 9.7130 0.0030 0.0110 0.0050
sgd_update 17020 170.3710 0.0020 0.0790 0.0100
_backward_Activation 6808 503.5580 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66648 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5832 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 851 3388.7241 0.3060 73.5870 3.9820
MXNDArrayCreateEx 972 33.6970 0.0310 0.0640 0.0347
MXNDArrayWaitAll 852 18494.0215 5.6130 25.0610 21.7066
MXNDArraySyncCopyToCPU 1824 14319.0381 0.0600 24.1980 7.8503
MXNDArrayGetDType 1824 0.6660 0.0000 0.0010 0.0004
MXNDArrayGetShape 4620 2.1620 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1944 17.1240 0.0070 0.0280 0.0088
MXNDArrayFree 21756 223.6850 0.0000 0.9140 0.0103
MXNet C API Calls 97764 585.1210 487.3580 585.1210 48.8815
MXNet C API Concurrency 195527 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1704 0.8120 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8520 4.4130 0.0000 0.0230 0.0005
MXNDArraySetGradState 8520 4.3530 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1704 0.7250 0.0000 0.0010 0.0004
MXAutogradBackwardEx 852 471.8480 0.5250 0.9450 0.5538
MXNDArrayGetContext 13488 5.9210 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28332 729.7850 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3408 8331.6963 0.7010 3.7500 2.4447
FullyConnected 5832 563.1560 0.0080 0.3430 0.0966
argmax 1944 38.5550 0.0050 0.0480 0.0198
_backward_pick 1704 27.0720 0.0130 0.0320 0.0159
_backward_mul_scalar 1704 7.4530 0.0030 0.0220 0.0044
DeleteVariable 72480 515.7570 0.0010 0.9010 0.0071
Flatten 1944 81.2220 0.0080 0.1120 0.0418
Activation 7776 361.7800 0.0030 1.9440 0.0465
SetValueOp 1704 11.0720 0.0050 0.0180 0.0065
pick 1704 13.4160 0.0060 0.0280 0.0079
Pooling 3888 5407.8472 0.0640 3.1720 1.3909
Convolution 3888 22082.4414 0.2260 13.2050 5.6796
CopyCPU2CPU 1944 11.0850 0.0030 0.0360 0.0057
log_softmax 1704 53.1330 0.0150 0.0550 0.0312
_backward_copy 1704 88.2780 0.0220 0.0760 0.0518
_backward_Convolution 3408 31516.7051 2.6240 12.5610 9.2479
_mul_scalar 1704 6.0810 0.0020 0.0230 0.0036
_backward_log_softmax 1704 29.1210 0.0090 0.0400 0.0171
mean 5352 102.5390 0.0080 0.0730 0.0192
_backward_mean 1704 18.7220 0.0080 0.1030 0.0110
_backward_FullyConnected 5112 981.0940 0.0280 0.5940 0.1919
WaitForVar 3648 32.4530 0.0060 0.0280 0.0089
broadcast_equal 1944 9.7220 0.0030 0.0110 0.0050
sgd_update 17040 170.6450 0.0020 0.0790 0.0100
_backward_Activation 6816 504.1500 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66722 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5838 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 852 3392.6211 0.3060 73.5870 3.9819
MXNDArrayCreateEx 973 33.7300 0.0310 0.0640 0.0347
MXNDArrayWaitAll 853 18515.8262 5.6130 25.0610 21.7067
MXNDArraySyncCopyToCPU 1826 14333.4082 0.0600 24.1980 7.8496
MXNDArrayGetDType 1826 0.6670 0.0000 0.0010 0.0004
MXNDArrayGetShape 4625 2.1640 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1946 17.1410 0.0070 0.0280 0.0088
MXNDArrayFree 21779 223.9170 0.0000 0.9140 0.0103
MXNet C API Calls 97871 585.2280 487.3580 585.2280 48.9350
MXNet C API Concurrency 195741 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1706 0.8120 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8530 4.4210 0.0000 0.0230 0.0005
MXNDArraySetGradState 8530 4.3570 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1706 0.7260 0.0000 0.0010 0.0004
MXAutogradBackwardEx 853 472.3800 0.5250 0.9450 0.5538
MXNDArrayGetContext 13502 5.9260 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28363 730.5570 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3412 8341.3145 0.7010 3.7500 2.4447
FullyConnected 5838 563.6820 0.0080 0.3430 0.0966
argmax 1946 38.5920 0.0050 0.0480 0.0198
_backward_pick 1706 27.1040 0.0130 0.0320 0.0159
_backward_mul_scalar 1706 7.4620 0.0030 0.0220 0.0044
DeleteVariable 72560 516.2680 0.0010 0.9010 0.0071
Flatten 1946 81.3010 0.0080 0.1120 0.0418
Activation 7784 362.1490 0.0030 1.9440 0.0465
SetValueOp 1706 11.0850 0.0050 0.0180 0.0065
pick 1706 13.4320 0.0060 0.0280 0.0079
Pooling 3892 5413.2002 0.0640 3.1720 1.3909
Convolution 3892 22104.6035 0.2260 13.2050 5.6795
CopyCPU2CPU 1946 11.0950 0.0030 0.0360 0.0057
log_softmax 1706 53.2280 0.0150 0.0550 0.0312
_backward_copy 1706 88.3830 0.0220 0.0760 0.0518
_backward_Convolution 3412 31553.7207 2.6240 12.5610 9.2479
_mul_scalar 1706 6.0880 0.0020 0.0230 0.0036
_backward_log_softmax 1706 29.1530 0.0090 0.0400 0.0171
mean 5358 102.6600 0.0080 0.0730 0.0192
_backward_mean 1706 18.7440 0.0080 0.1030 0.0110
_backward_FullyConnected 5118 982.1730 0.0280 0.5940 0.1919
WaitForVar 3652 32.4900 0.0060 0.0280 0.0089
broadcast_equal 1946 9.7310 0.0030 0.0110 0.0050
sgd_update 17060 170.8490 0.0020 0.0790 0.0100
_backward_Activation 6824 504.7170 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66796 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5844 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 853 3396.5171 0.3060 73.5870 3.9818
MXNDArrayCreateEx 974 33.7630 0.0310 0.0640 0.0347
MXNDArrayWaitAll 854 18537.6543 5.6130 25.0610 21.7069
MXNDArraySyncCopyToCPU 1828 14347.8184 0.0600 24.1980 7.8489
MXNDArrayGetDType 1828 0.6680 0.0000 0.0010 0.0004
MXNDArrayGetShape 4630 2.1650 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1948 17.1570 0.0070 0.0280 0.0088
MXNDArrayFree 21802 224.1480 0.0000 0.9140 0.0103
MXNet C API Calls 97978 585.3350 487.3580 585.3350 48.9885
MXNet C API Concurrency 195955 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1708 0.8130 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8540 4.4240 0.0000 0.0230 0.0005
MXNDArraySetGradState 8540 4.3630 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1708 0.7280 0.0000 0.0010 0.0004
MXAutogradBackwardEx 854 472.9310 0.5250 0.9450 0.5538
MXNDArrayGetContext 13516 5.9340 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28394 731.3270 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3416 8351.0117 0.7010 3.7500 2.4447
FullyConnected 5844 564.2270 0.0080 0.3430 0.0965
argmax 1948 38.6300 0.0050 0.0480 0.0198
_backward_pick 1708 27.1350 0.0130 0.0320 0.0159
_backward_mul_scalar 1708 7.4710 0.0030 0.0220 0.0044
DeleteVariable 72640 516.8110 0.0010 0.9010 0.0071
Flatten 1948 81.3800 0.0080 0.1120 0.0418
Activation 7792 362.5170 0.0030 1.9440 0.0465
SetValueOp 1708 11.0980 0.0050 0.0180 0.0065
pick 1708 13.4480 0.0060 0.0280 0.0079
Pooling 3896 5418.5669 0.0640 3.1720 1.3908
Convolution 3896 22126.8086 0.2260 13.2050 5.6794
CopyCPU2CPU 1948 11.1060 0.0030 0.0360 0.0057
log_softmax 1708 53.2890 0.0150 0.0550 0.0312
_backward_copy 1708 88.4890 0.0220 0.0760 0.0518
_backward_Convolution 3416 31590.7637 2.6240 12.5610 9.2479
_mul_scalar 1708 6.0940 0.0020 0.0230 0.0036
_backward_log_softmax 1708 29.1850 0.0090 0.0400 0.0171
mean 5364 102.7780 0.0080 0.0730 0.0192
_backward_mean 1708 18.7660 0.0080 0.1030 0.0110
_backward_FullyConnected 5124 983.2470 0.0280 0.5940 0.1919
WaitForVar 3656 32.5250 0.0060 0.0280 0.0089
broadcast_equal 1948 9.7400 0.0030 0.0110 0.0050
sgd_update 17080 170.9980 0.0020 0.0790 0.0100
_backward_Activation 6832 505.3480 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66870 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5850 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 854 3400.4241 0.3060 73.5870 3.9818
MXNDArrayCreateEx 975 33.7960 0.0310 0.0640 0.0347
MXNDArrayWaitAll 855 18559.4258 5.6130 25.0610 21.7069
MXNDArraySyncCopyToCPU 1830 14362.2764 0.0600 24.1980 7.8482
MXNDArrayGetDType 1830 0.6690 0.0000 0.0010 0.0004
MXNDArrayGetShape 4635 2.1660 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1950 17.1750 0.0070 0.0280 0.0088
MXNDArrayFree 21825 224.3690 0.0000 0.9140 0.0103
MXNet C API Calls 98085 585.4420 487.3580 585.4420 49.0420
MXNet C API Concurrency 196169 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1710 0.8130 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8550 4.4290 0.0000 0.0230 0.0005
MXNDArraySetGradState 8550 4.3660 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1710 0.7290 0.0000 0.0010 0.0004
MXAutogradBackwardEx 855 473.4600 0.5250 0.9450 0.5538
MXNDArrayGetContext 13530 5.9380 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28425 732.0850 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3420 8360.7939 0.7010 3.7500 2.4447
FullyConnected 5850 564.7690 0.0080 0.3430 0.0965
argmax 1950 38.6680 0.0050 0.0480 0.0198
_backward_pick 1710 27.1660 0.0130 0.0320 0.0159
_backward_mul_scalar 1710 7.4790 0.0030 0.0220 0.0044
DeleteVariable 72720 517.3310 0.0010 0.9010 0.0071
Flatten 1950 81.4610 0.0080 0.1120 0.0418
Activation 7800 362.8870 0.0030 1.9440 0.0465
SetValueOp 1710 11.1110 0.0050 0.0180 0.0065
pick 1710 13.4630 0.0060 0.0280 0.0079
Pooling 3900 5423.9238 0.0640 3.1720 1.3907
Convolution 3900 22148.9961 0.2260 13.2050 5.6792
CopyCPU2CPU 1950 11.1160 0.0030 0.0360 0.0057
log_softmax 1710 53.3480 0.0150 0.0550 0.0312
_backward_copy 1710 88.5940 0.0220 0.0760 0.0518
_backward_Convolution 3420 31627.5938 2.6240 12.5610 9.2478
_mul_scalar 1710 6.1010 0.0020 0.0230 0.0036
_backward_log_softmax 1710 29.2170 0.0090 0.0400 0.0171
mean 5370 102.9190 0.0080 0.0730 0.0192
_backward_mean 1710 18.7880 0.0080 0.1030 0.0110
_backward_FullyConnected 5130 984.3590 0.0280 0.5940 0.1919
WaitForVar 3660 32.5600 0.0060 0.0280 0.0089
broadcast_equal 1950 9.7500 0.0030 0.0110 0.0050
sgd_update 17100 171.1990 0.0020 0.0790 0.0100
_backward_Activation 6840 505.9190 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 66944 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5856 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 855 3404.3040 0.3060 73.5870 3.9816
MXNDArrayCreateEx 976 33.8300 0.0310 0.0640 0.0347
MXNDArrayWaitAll 856 18581.1133 5.6130 25.0610 21.7069
MXNDArraySyncCopyToCPU 1832 14377.0479 0.0600 24.1980 7.8477
MXNDArrayGetDType 1832 0.6690 0.0000 0.0010 0.0004
MXNDArrayGetShape 4640 2.1670 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1952 17.1920 0.0070 0.0280 0.0088
MXNDArrayFree 21848 224.5940 0.0000 0.9140 0.0103
MXNet C API Calls 98192 585.5490 487.3580 585.5490 49.0955
MXNet C API Concurrency 196383 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1712 0.8150 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8560 4.4310 0.0000 0.0230 0.0005
MXNDArraySetGradState 8560 4.3670 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1712 0.7300 0.0000 0.0010 0.0004
MXAutogradBackwardEx 856 474.0120 0.5250 0.9450 0.5538
MXNDArrayGetContext 13544 5.9440 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28456 732.8520 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3424 8370.5039 0.7010 3.7500 2.4447
FullyConnected 5856 565.5270 0.0080 0.3430 0.0966
argmax 1952 38.7100 0.0050 0.0480 0.0198
_backward_pick 1712 27.1970 0.0130 0.0320 0.0159
_backward_mul_scalar 1712 7.4870 0.0030 0.0220 0.0044
DeleteVariable 72800 517.8640 0.0010 0.9010 0.0071
Flatten 1952 81.5390 0.0080 0.1120 0.0418
Activation 7808 363.2680 0.0030 1.9440 0.0465
SetValueOp 1712 11.1230 0.0050 0.0180 0.0065
pick 1712 13.4790 0.0060 0.0280 0.0079
Pooling 3904 5429.2729 0.0640 3.1720 1.3907
Convolution 3904 22171.2090 0.2260 13.2050 5.6791
CopyCPU2CPU 1952 11.1260 0.0030 0.0360 0.0057
log_softmax 1712 53.4200 0.0150 0.0550 0.0312
_backward_copy 1712 88.6960 0.0220 0.0760 0.0518
_backward_Convolution 3424 31664.2852 2.6240 12.5610 9.2477
_mul_scalar 1712 6.1080 0.0020 0.0230 0.0036
_backward_log_softmax 1712 29.2590 0.0090 0.0400 0.0171
mean 5376 103.0260 0.0080 0.0730 0.0192
_backward_mean 1712 18.8090 0.0080 0.1030 0.0110
_backward_FullyConnected 5136 985.7750 0.0280 0.5940 0.1919
WaitForVar 3664 32.5950 0.0060 0.0280 0.0089
broadcast_equal 1952 9.7600 0.0030 0.0110 0.0050
sgd_update 17120 171.4720 0.0020 0.0790 0.0100
_backward_Activation 6848 506.5230 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67018 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5862 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 856 3408.1960 0.3060 73.5870 3.9815
MXNDArrayCreateEx 977 33.8640 0.0310 0.0640 0.0347
MXNDArrayWaitAll 857 18602.9648 5.6130 25.0610 21.7071
MXNDArraySyncCopyToCPU 1834 14391.5234 0.0600 24.1980 7.8471
MXNDArrayGetDType 1834 0.6710 0.0000 0.0010 0.0004
MXNDArrayGetShape 4645 2.1700 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1954 17.2080 0.0070 0.0280 0.0088
MXNDArrayFree 21871 224.8130 0.0000 0.9140 0.0103
MXNet C API Calls 98299 585.6560 487.3580 585.6560 49.1490
MXNet C API Concurrency 196597 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1714 0.8160 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8570 4.4360 0.0000 0.0230 0.0005
MXNDArraySetGradState 8570 4.3710 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1714 0.7300 0.0000 0.0010 0.0004
MXAutogradBackwardEx 857 474.5420 0.5250 0.9450 0.5537
MXNDArrayGetContext 13558 5.9530 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28487 733.6120 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3428 8380.3154 0.7010 3.7500 2.4447
FullyConnected 5862 566.0630 0.0080 0.3430 0.0966
argmax 1954 38.7490 0.0050 0.0480 0.0198
_backward_pick 1714 27.2270 0.0130 0.0320 0.0159
_backward_mul_scalar 1714 7.4960 0.0030 0.0220 0.0044
DeleteVariable 72880 518.3740 0.0010 0.9010 0.0071
Flatten 1954 81.6160 0.0080 0.1120 0.0418
Activation 7816 363.6360 0.0030 1.9440 0.0465
SetValueOp 1714 11.1370 0.0050 0.0180 0.0065
pick 1714 13.4950 0.0060 0.0280 0.0079
Pooling 3908 5434.6382 0.0640 3.1720 1.3906
Convolution 3908 22193.3809 0.2260 13.2050 5.6790
CopyCPU2CPU 1954 11.1370 0.0030 0.0360 0.0057
log_softmax 1714 53.4790 0.0150 0.0550 0.0312
_backward_copy 1714 88.8020 0.0220 0.0760 0.0518
_backward_Convolution 3428 31701.3027 2.6240 12.5610 9.2478
_mul_scalar 1714 6.1150 0.0020 0.0230 0.0036
_backward_log_softmax 1714 29.2910 0.0090 0.0400 0.0171
mean 5382 103.1410 0.0080 0.0730 0.0192
_backward_mean 1714 18.8300 0.0080 0.1030 0.0110
_backward_FullyConnected 5142 986.8520 0.0280 0.5940 0.1919
WaitForVar 3668 32.6300 0.0060 0.0280 0.0089
broadcast_equal 1954 9.7700 0.0030 0.0110 0.0050
sgd_update 17140 171.6730 0.0020 0.0790 0.0100
_backward_Activation 6856 507.0940 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67092 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5868 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 857 3412.0950 0.3060 73.5870 3.9814
MXNDArrayCreateEx 978 33.8970 0.0310 0.0640 0.0347
MXNDArrayWaitAll 858 18624.8730 5.6130 25.0610 21.7073
MXNDArraySyncCopyToCPU 1836 14405.9727 0.0600 24.1980 7.8464
MXNDArrayGetDType 1836 0.6720 0.0000 0.0010 0.0004
MXNDArrayGetShape 4650 2.1720 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1956 17.2240 0.0070 0.0280 0.0088
MXNDArrayFree 21894 225.0450 0.0000 0.9140 0.0103
MXNet C API Calls 98406 585.7630 487.3580 585.7630 49.2025
MXNet C API Concurrency 196811 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1716 0.8170 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8580 4.4390 0.0000 0.0230 0.0005
MXNDArraySetGradState 8580 4.3770 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1716 0.7310 0.0000 0.0010 0.0004
MXAutogradBackwardEx 858 475.0710 0.5250 0.9450 0.5537
MXNDArrayGetContext 13572 5.9590 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28518 734.3830 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3432 8390.0723 0.7010 3.7500 2.4447
FullyConnected 5868 566.5840 0.0080 0.3430 0.0966
argmax 1956 38.7870 0.0050 0.0480 0.0198
_backward_pick 1716 27.2580 0.0130 0.0320 0.0159
_backward_mul_scalar 1716 7.5050 0.0030 0.0220 0.0044
DeleteVariable 72960 518.9270 0.0010 0.9010 0.0071
Flatten 1956 81.6950 0.0080 0.1120 0.0418
Activation 7824 364.0050 0.0030 1.9440 0.0465
SetValueOp 1716 11.1510 0.0050 0.0180 0.0065
pick 1716 13.5110 0.0060 0.0280 0.0079
Pooling 3912 5440.0142 0.0640 3.1720 1.3906
Convolution 3912 22215.6230 0.2260 13.2050 5.6788
CopyCPU2CPU 1956 11.1470 0.0030 0.0360 0.0057
log_softmax 1716 53.5370 0.0150 0.0550 0.0312
_backward_copy 1716 88.9070 0.0220 0.0760 0.0518
_backward_Convolution 3432 31738.4141 2.6240 12.5610 9.2478
_mul_scalar 1716 6.1220 0.0020 0.0230 0.0036
_backward_log_softmax 1716 29.3230 0.0090 0.0400 0.0171
mean 5388 103.2600 0.0080 0.0730 0.0192
_backward_mean 1716 18.8510 0.0080 0.1030 0.0110
_backward_FullyConnected 5148 987.9220 0.0280 0.5940 0.1919
WaitForVar 3672 32.6670 0.0060 0.0280 0.0089
broadcast_equal 1956 9.7800 0.0030 0.0110 0.0050
sgd_update 17160 171.8760 0.0020 0.0790 0.0100
_backward_Activation 6864 507.6790 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67166 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5874 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 858 3415.9951 0.3060 73.5870 3.9813
MXNDArrayCreateEx 979 33.9300 0.0310 0.0640 0.0347
MXNDArrayWaitAll 859 18646.5645 5.6130 25.0610 21.7073
MXNDArraySyncCopyToCPU 1838 14420.4150 0.0600 24.1980 7.8457
MXNDArrayGetDType 1838 0.6720 0.0000 0.0010 0.0004
MXNDArrayGetShape 4655 2.1740 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1958 17.2410 0.0070 0.0280 0.0088
MXNDArrayFree 21917 225.2750 0.0000 0.9140 0.0103
MXNet C API Calls 98513 585.8700 487.3580 585.8700 49.2560
MXNet C API Concurrency 197025 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1718 0.8190 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8590 4.4480 0.0000 0.0230 0.0005
MXNDArraySetGradState 8590 4.3820 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1718 0.7320 0.0000 0.0010 0.0004
MXAutogradBackwardEx 859 475.6040 0.5250 0.9450 0.5537
MXNDArrayGetContext 13586 5.9660 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28549 735.1850 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3436 8399.7783 0.7010 3.7500 2.4446
FullyConnected 5874 567.1220 0.0080 0.3430 0.0965
argmax 1958 38.8260 0.0050 0.0480 0.0198
_backward_pick 1718 27.2890 0.0130 0.0320 0.0159
_backward_mul_scalar 1718 7.5140 0.0030 0.0220 0.0044
DeleteVariable 73040 519.4640 0.0010 0.9010 0.0071
Flatten 1958 81.7760 0.0080 0.1120 0.0418
Activation 7832 364.3760 0.0030 1.9440 0.0465
SetValueOp 1718 11.1640 0.0050 0.0180 0.0065
pick 1718 13.5260 0.0060 0.0280 0.0079
Pooling 3916 5445.3940 0.0640 3.1720 1.3906
Convolution 3916 22237.8379 0.2260 13.2050 5.6787
CopyCPU2CPU 1958 11.1560 0.0030 0.0360 0.0057
log_softmax 1718 53.5960 0.0150 0.0550 0.0312
_backward_copy 1718 89.0120 0.0220 0.0760 0.0518
_backward_Convolution 3436 31775.2051 2.6240 12.5610 9.2477
_mul_scalar 1718 6.1290 0.0020 0.0230 0.0036
_backward_log_softmax 1718 29.3550 0.0090 0.0400 0.0171
mean 5394 103.3750 0.0080 0.0730 0.0192
_backward_mean 1718 18.8720 0.0080 0.1030 0.0110
_backward_FullyConnected 5154 988.9980 0.0280 0.5940 0.1919
WaitForVar 3676 32.7030 0.0060 0.0280 0.0089
broadcast_equal 1958 9.7900 0.0030 0.0110 0.0050
sgd_update 17180 172.0260 0.0020 0.0790 0.0100
_backward_Activation 6872 508.3030 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67240 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5880 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 859 3419.8911 0.3060 73.5870 3.9812
MXNDArrayCreateEx 980 33.9640 0.0310 0.0640 0.0347
MXNDArrayWaitAll 860 18668.2754 5.6130 25.0610 21.7073
MXNDArraySyncCopyToCPU 1840 14435.2158 0.0600 24.1980 7.8452
MXNDArrayGetDType 1840 0.6730 0.0000 0.0010 0.0004
MXNDArrayGetShape 4660 2.1740 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1960 17.2580 0.0070 0.0280 0.0088
MXNDArrayFree 21940 225.5010 0.0000 0.9140 0.0103
MXNet C API Calls 98620 585.9770 487.3580 585.9770 49.3095
MXNet C API Concurrency 197239 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1720 0.8200 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8600 4.4500 0.0000 0.0230 0.0005
MXNDArraySetGradState 8600 4.3850 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1720 0.7320 0.0000 0.0010 0.0004
MXAutogradBackwardEx 860 476.1350 0.5250 0.9450 0.5536
MXNDArrayGetContext 13600 5.9730 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28580 735.9500 0.0080 0.2420 0.0258
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3440 8409.5264 0.7010 3.7500 2.4446
FullyConnected 5880 567.8740 0.0080 0.3430 0.0966
argmax 1960 38.8700 0.0050 0.0480 0.0198
_backward_pick 1720 27.3220 0.0130 0.0320 0.0159
_backward_mul_scalar 1720 7.5230 0.0030 0.0220 0.0044
DeleteVariable 73120 519.9950 0.0010 0.9010 0.0071
Flatten 1960 81.8550 0.0080 0.1120 0.0418
Activation 7840 364.7560 0.0030 1.9440 0.0465
SetValueOp 1720 11.1780 0.0050 0.0180 0.0065
pick 1720 13.5430 0.0060 0.0280 0.0079
Pooling 3920 5450.7412 0.0640 3.1720 1.3905
Convolution 3920 22260.0625 0.2260 13.2050 5.6786
CopyCPU2CPU 1960 11.1660 0.0030 0.0360 0.0057
log_softmax 1720 53.6680 0.0150 0.0550 0.0312
_backward_copy 1720 89.1140 0.0220 0.0760 0.0518
_backward_Convolution 3440 31811.9434 2.6240 12.5610 9.2477
_mul_scalar 1720 6.1360 0.0020 0.0230 0.0036
_backward_log_softmax 1720 29.3980 0.0090 0.0400 0.0171
mean 5400 103.5300 0.0080 0.0730 0.0192
_backward_mean 1720 18.8940 0.0080 0.1030 0.0110
_backward_FullyConnected 5160 990.4140 0.0280 0.5940 0.1919
WaitForVar 3680 32.7380 0.0060 0.0280 0.0089
broadcast_equal 1960 9.7990 0.0030 0.0110 0.0050
sgd_update 17200 172.3030 0.0020 0.0790 0.0100
_backward_Activation 6880 508.8750 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67314 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5886 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 860 3422.4910 0.3060 73.5870 3.9796
MXNDArrayCreateEx 981 33.9970 0.0310 0.0640 0.0347
MXNDArrayWaitAll 861 18690.1406 5.6130 25.0610 21.7075
MXNDArraySyncCopyToCPU 1842 14449.6396 0.0600 24.1980 7.8445
MXNDArrayGetDType 1842 0.6730 0.0000 0.0010 0.0004
MXNDArrayGetShape 4665 2.1740 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1962 17.2760 0.0070 0.0280 0.0088
MXNDArrayFree 21963 225.7270 0.0000 0.9140 0.0103
MXNet C API Calls 98727 586.0840 487.3580 586.0840 49.3630
MXNet C API Concurrency 197453 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1722 0.8220 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8610 4.4560 0.0000 0.0230 0.0005
MXNDArraySetGradState 8610 4.3870 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1722 0.7330 0.0000 0.0010 0.0004
MXAutogradBackwardEx 861 476.6690 0.5250 0.9450 0.5536
MXNDArrayGetContext 13614 5.9790 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28611 736.7200 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3444 8419.2881 0.7010 3.7500 2.4446
FullyConnected 5886 568.4180 0.0080 0.3430 0.0966
argmax 1962 38.9080 0.0050 0.0480 0.0198
_backward_pick 1722 27.3530 0.0130 0.0320 0.0159
_backward_mul_scalar 1722 7.5320 0.0030 0.0220 0.0044
DeleteVariable 73200 520.4990 0.0010 0.9010 0.0071
Flatten 1962 81.9330 0.0080 0.1120 0.0418
Activation 7848 365.1250 0.0030 1.9440 0.0465
SetValueOp 1722 11.1910 0.0050 0.0180 0.0065
pick 1722 13.5580 0.0060 0.0280 0.0079
Pooling 3924 5456.1182 0.0640 3.1720 1.3904
Convolution 3924 22282.2402 0.2260 13.2050 5.6785
CopyCPU2CPU 1962 11.1750 0.0030 0.0360 0.0057
log_softmax 1722 53.7270 0.0150 0.0550 0.0312
_backward_copy 1722 89.2200 0.0220 0.0760 0.0518
_backward_Convolution 3444 31848.9805 2.6240 12.5610 9.2477
_mul_scalar 1722 6.1440 0.0020 0.0230 0.0036
_backward_log_softmax 1722 29.4300 0.0090 0.0400 0.0171
mean 5406 103.6470 0.0080 0.0730 0.0192
_backward_mean 1722 18.9160 0.0080 0.1030 0.0110
_backward_FullyConnected 5166 991.4850 0.0280 0.5940 0.1919
WaitForVar 3684 32.7740 0.0060 0.0280 0.0089
broadcast_equal 1962 9.8090 0.0030 0.0110 0.0050
sgd_update 17220 172.4500 0.0020 0.0790 0.0100
_backward_Activation 6888 509.5120 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67388 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5892 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 861 3426.4041 0.3060 73.5870 3.9796
MXNDArrayCreateEx 982 34.0300 0.0310 0.0640 0.0347
MXNDArrayWaitAll 862 18711.9922 5.6130 25.0610 21.7076
MXNDArraySyncCopyToCPU 1844 14464.0986 0.0600 24.1980 7.8439
MXNDArrayGetDType 1844 0.6730 0.0000 0.0010 0.0004
MXNDArrayGetShape 4670 2.1760 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1964 17.2920 0.0070 0.0280 0.0088
MXNDArrayFree 21986 225.9540 0.0000 0.9140 0.0103
MXNet C API Calls 98834 586.1910 487.3580 586.1910 49.4165
MXNet C API Concurrency 197667 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1724 0.8230 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8620 4.4590 0.0000 0.0230 0.0005
MXNDArraySetGradState 8620 4.3900 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1724 0.7330 0.0000 0.0010 0.0004
MXAutogradBackwardEx 862 477.2000 0.5250 0.9450 0.5536
MXNDArrayGetContext 13628 5.9860 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28642 737.5020 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3448 8429.0371 0.7010 3.7500 2.4446
FullyConnected 5892 568.9530 0.0080 0.3430 0.0966
argmax 1964 38.9460 0.0050 0.0480 0.0198
_backward_pick 1724 27.3830 0.0130 0.0320 0.0159
_backward_mul_scalar 1724 7.5400 0.0030 0.0220 0.0044
DeleteVariable 73280 521.0280 0.0010 0.9010 0.0071
Flatten 1964 82.0110 0.0080 0.1120 0.0418
Activation 7856 365.4980 0.0030 1.9440 0.0465
SetValueOp 1724 11.2040 0.0050 0.0180 0.0065
pick 1724 13.5720 0.0060 0.0280 0.0079
Pooling 3928 5461.4888 0.0640 3.1720 1.3904
Convolution 3928 22304.4863 0.2260 13.2050 5.6783
CopyCPU2CPU 1964 11.1840 0.0030 0.0360 0.0057
log_softmax 1724 53.7880 0.0150 0.0550 0.0312
_backward_copy 1724 89.3250 0.0220 0.0760 0.0518
_backward_Convolution 3448 31886.0625 2.6240 12.5610 9.2477
_mul_scalar 1724 6.1500 0.0020 0.0230 0.0036
_backward_log_softmax 1724 29.4610 0.0090 0.0400 0.0171
mean 5412 103.7610 0.0080 0.0730 0.0192
_backward_mean 1724 18.9380 0.0080 0.1030 0.0110
_backward_FullyConnected 5172 992.5520 0.0280 0.5940 0.1919
WaitForVar 3688 32.8100 0.0060 0.0280 0.0089
broadcast_equal 1964 9.8180 0.0030 0.0110 0.0050
sgd_update 17240 172.5990 0.0020 0.0790 0.0100
_backward_Activation 6896 510.1400 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67462 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5898 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 862 3430.3040 0.3060 73.5870 3.9795
MXNDArrayCreateEx 983 34.0630 0.0310 0.0640 0.0347
MXNDArrayWaitAll 863 18733.6152 5.6130 25.0610 21.7076
MXNDArraySyncCopyToCPU 1846 14478.5322 0.0600 24.1980 7.8432
MXNDArrayGetDType 1846 0.6740 0.0000 0.0010 0.0004
MXNDArrayGetShape 4675 2.1780 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1966 17.3090 0.0070 0.0280 0.0088
MXNDArrayFree 22009 226.1840 0.0000 0.9140 0.0103
MXNet C API Calls 98941 586.2980 487.3580 586.2980 49.4700
MXNet C API Concurrency 197881 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1726 0.8230 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8630 4.4650 0.0000 0.0230 0.0005
MXNDArraySetGradState 8630 4.3950 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1726 0.7350 0.0000 0.0010 0.0004
MXAutogradBackwardEx 863 477.7280 0.5250 0.9450 0.5536
MXNDArrayGetContext 13642 5.9910 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28673 738.2890 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3452 8438.8027 0.7010 3.7500 2.4446
FullyConnected 5898 569.5210 0.0080 0.3430 0.0966
argmax 1966 38.9840 0.0050 0.0480 0.0198
_backward_pick 1726 27.4140 0.0130 0.0320 0.0159
_backward_mul_scalar 1726 7.5480 0.0030 0.0220 0.0044
DeleteVariable 73360 521.5660 0.0010 0.9010 0.0071
Flatten 1966 82.0900 0.0080 0.1120 0.0418
Activation 7864 365.8500 0.0030 1.9440 0.0465
SetValueOp 1726 11.2160 0.0050 0.0180 0.0065
pick 1726 13.5880 0.0060 0.0280 0.0079
Pooling 3932 5466.8589 0.0640 3.1720 1.3904
Convolution 3932 22326.6836 0.2260 13.2050 5.6782
CopyCPU2CPU 1966 11.1940 0.0030 0.0360 0.0057
log_softmax 1726 53.8490 0.0150 0.0550 0.0312
_backward_copy 1726 89.4270 0.0220 0.0760 0.0518
_backward_Convolution 3452 31922.9121 2.6240 12.5610 9.2477
_mul_scalar 1726 6.1580 0.0020 0.0230 0.0036
_backward_log_softmax 1726 29.4940 0.0090 0.0400 0.0171
mean 5418 103.8760 0.0080 0.0730 0.0192
_backward_mean 1726 18.9590 0.0080 0.1030 0.0110
_backward_FullyConnected 5178 993.6210 0.0280 0.5940 0.1919
WaitForVar 3692 32.8440 0.0060 0.0280 0.0089
broadcast_equal 1966 9.8270 0.0030 0.0110 0.0050
sgd_update 17260 172.8000 0.0020 0.0790 0.0100
_backward_Activation 6904 510.6630 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67536 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5904 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 863 3434.2129 0.3060 73.5870 3.9794
MXNDArrayCreateEx 984 34.0960 0.0310 0.0640 0.0347
MXNDArrayWaitAll 864 18755.3203 5.6130 25.0610 21.7075
MXNDArraySyncCopyToCPU 1848 14493.3076 0.0600 24.1980 7.8427
MXNDArrayGetDType 1848 0.6740 0.0000 0.0010 0.0004
MXNDArrayGetShape 4680 2.1800 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1968 17.3260 0.0070 0.0280 0.0088
MXNDArrayFree 22032 226.4130 0.0000 0.9140 0.0103
MXNet C API Calls 99048 586.4050 487.3580 586.4050 49.5235
MXNet C API Concurrency 198095 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1728 0.8240 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8640 4.4690 0.0000 0.0230 0.0005
MXNDArraySetGradState 8640 4.3970 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1728 0.7370 0.0000 0.0010 0.0004
MXAutogradBackwardEx 864 478.2600 0.5250 0.9450 0.5535
MXNDArrayGetContext 13656 6.0010 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28704 739.0710 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3456 8448.5566 0.7010 3.7500 2.4446
FullyConnected 5904 570.2490 0.0080 0.3430 0.0966
argmax 1968 39.0270 0.0050 0.0480 0.0198
_backward_pick 1728 27.4460 0.0130 0.0320 0.0159
_backward_mul_scalar 1728 7.5570 0.0030 0.0220 0.0044
DeleteVariable 73440 522.0820 0.0010 0.9010 0.0071
Flatten 1968 82.1690 0.0080 0.1120 0.0418
Activation 7872 366.2290 0.0030 1.9440 0.0465
SetValueOp 1728 11.2290 0.0050 0.0180 0.0065
pick 1728 13.6030 0.0060 0.0280 0.0079
Pooling 3936 5472.2432 0.0640 3.1720 1.3903
Convolution 3936 22348.9355 0.2260 13.2050 5.6781
CopyCPU2CPU 1968 11.2040 0.0030 0.0360 0.0057
log_softmax 1728 53.9210 0.0150 0.0550 0.0312
_backward_copy 1728 89.5270 0.0220 0.0760 0.0518
_backward_Convolution 3456 31959.6738 2.6240 12.5610 9.2476
_mul_scalar 1728 6.1650 0.0020 0.0230 0.0036
_backward_log_softmax 1728 29.5350 0.0090 0.0400 0.0171
mean 5424 103.9810 0.0080 0.0730 0.0192
_backward_mean 1728 18.9800 0.0080 0.1030 0.0110
_backward_FullyConnected 5184 995.0300 0.0280 0.5940 0.1919
WaitForVar 3696 32.8800 0.0060 0.0280 0.0089
broadcast_equal 1968 9.8370 0.0030 0.0110 0.0050
sgd_update 17280 173.0710 0.0020 0.0790 0.0100
_backward_Activation 6912 511.2400 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67610 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5910 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 864 3438.1240 0.3060 73.5870 3.9793
MXNDArrayCreateEx 985 34.1290 0.0310 0.0640 0.0346
MXNDArrayWaitAll 865 18777.2344 5.6130 25.0610 21.7078
MXNDArraySyncCopyToCPU 1850 14507.6963 0.0600 24.1980 7.8420
MXNDArrayGetDType 1850 0.6740 0.0000 0.0010 0.0004
MXNDArrayGetShape 4685 2.1830 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1970 17.3420 0.0070 0.0280 0.0088
MXNDArrayFree 22055 226.6460 0.0000 0.9140 0.0103
MXNet C API Calls 99155 586.5120 487.3580 586.5120 49.5770
MXNet C API Concurrency 198309 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1730 0.8250 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8650 4.4730 0.0000 0.0230 0.0005
MXNDArraySetGradState 8650 4.4030 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1730 0.7370 0.0000 0.0010 0.0004
MXAutogradBackwardEx 865 478.7920 0.5250 0.9450 0.5535
MXNDArrayGetContext 13670 6.0090 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28735 739.8590 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3460 8458.3896 0.7010 3.7500 2.4446
FullyConnected 5910 570.7770 0.0080 0.3430 0.0966
argmax 1970 39.0650 0.0050 0.0480 0.0198
_backward_pick 1730 27.4760 0.0130 0.0320 0.0159
_backward_mul_scalar 1730 7.5650 0.0030 0.0220 0.0044
DeleteVariable 73520 522.6060 0.0010 0.9010 0.0071
Flatten 1970 82.2480 0.0080 0.1120 0.0418
Activation 7880 366.5820 0.0030 1.9440 0.0465
SetValueOp 1730 11.2410 0.0050 0.0180 0.0065
pick 1730 13.6180 0.0060 0.0280 0.0079
Pooling 3940 5477.5742 0.0640 3.1720 1.3902
Convolution 3940 22371.1035 0.2260 13.2050 5.6779
CopyCPU2CPU 1970 11.2150 0.0030 0.0360 0.0057
log_softmax 1730 53.9800 0.0150 0.0550 0.0312
_backward_copy 1730 89.6300 0.0220 0.0760 0.0518
_backward_Convolution 3460 31996.7578 2.6240 12.5610 9.2476
_mul_scalar 1730 6.1720 0.0020 0.0230 0.0036
_backward_log_softmax 1730 29.5660 0.0090 0.0400 0.0171
mean 5430 104.0940 0.0080 0.0730 0.0192
_backward_mean 1730 19.0020 0.0080 0.1030 0.0110
_backward_FullyConnected 5190 996.1450 0.0280 0.5940 0.1919
WaitForVar 3700 32.9160 0.0060 0.0280 0.0089
broadcast_equal 1970 9.8460 0.0030 0.0110 0.0050
sgd_update 17300 173.2700 0.0020 0.0790 0.0100
_backward_Activation 6920 511.7860 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67684 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5916 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 865 3442.0520 0.3060 73.5870 3.9793
MXNDArrayCreateEx 986 34.1620 0.0310 0.0640 0.0346
MXNDArrayWaitAll 866 18799.1172 5.6130 25.0610 21.7080
MXNDArraySyncCopyToCPU 1852 14522.1299 0.0600 24.1980 7.8413
MXNDArrayGetDType 1852 0.6750 0.0000 0.0010 0.0004
MXNDArrayGetShape 4690 2.1840 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1972 17.3580 0.0070 0.0280 0.0088
MXNDArrayFree 22078 226.8750 0.0000 0.9140 0.0103
MXNet C API Calls 99262 586.6190 487.3580 586.6190 49.6305
MXNet C API Concurrency 198523 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1732 0.8250 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8660 4.4790 0.0000 0.0230 0.0005
MXNDArraySetGradState 8660 4.4100 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1732 0.7370 0.0000 0.0010 0.0004
MXAutogradBackwardEx 866 479.3240 0.5250 0.9450 0.5535
MXNDArrayGetContext 13684 6.0150 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28766 740.6540 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3464 8468.2119 0.7010 3.7500 2.4446
FullyConnected 5916 571.2990 0.0080 0.3430 0.0966
argmax 1972 39.1020 0.0050 0.0480 0.0198
_backward_pick 1732 27.5060 0.0130 0.0320 0.0159
_backward_mul_scalar 1732 7.5730 0.0030 0.0220 0.0044
DeleteVariable 73600 523.1280 0.0010 0.9010 0.0071
Flatten 1972 82.3250 0.0080 0.1120 0.0417
Activation 7888 366.9340 0.0030 1.9440 0.0465
SetValueOp 1732 11.2540 0.0050 0.0180 0.0065
pick 1732 13.6320 0.0060 0.0280 0.0079
Pooling 3944 5482.9922 0.0640 3.1720 1.3902
Convolution 3944 22393.2949 0.2260 13.2050 5.6778
CopyCPU2CPU 1972 11.2250 0.0030 0.0360 0.0057
log_softmax 1732 54.0400 0.0150 0.0550 0.0312
_backward_copy 1732 89.7340 0.0220 0.0760 0.0518
_backward_Convolution 3464 32033.8184 2.6240 12.5610 9.2476
_mul_scalar 1732 6.1790 0.0020 0.0230 0.0036
_backward_log_softmax 1732 29.5970 0.0090 0.0400 0.0171
mean 5436 104.2060 0.0080 0.0730 0.0192
_backward_mean 1732 19.0230 0.0080 0.1030 0.0110
_backward_FullyConnected 5196 997.2100 0.0280 0.5940 0.1919
WaitForVar 3704 32.9520 0.0060 0.0280 0.0089
broadcast_equal 1972 9.8560 0.0030 0.0110 0.0050
sgd_update 17320 173.4700 0.0020 0.0790 0.0100
_backward_Activation 6928 512.3360 0.0070 0.3900 0.0740
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67758 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5922 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 866 3445.9541 0.3060 73.5870 3.9792
MXNDArrayCreateEx 987 34.1950 0.0310 0.0640 0.0346
MXNDArrayWaitAll 867 18820.7832 5.6130 25.0610 21.7079
MXNDArraySyncCopyToCPU 1854 14536.5244 0.0600 24.1980 7.8406
MXNDArrayGetDType 1854 0.6750 0.0000 0.0010 0.0004
MXNDArrayGetShape 4695 2.1860 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1974 17.3740 0.0070 0.0280 0.0088
MXNDArrayFree 22101 227.0990 0.0000 0.9140 0.0103
MXNet C API Calls 99369 586.7260 487.3580 586.7260 49.6840
MXNet C API Concurrency 198737 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1734 0.8250 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8670 4.4830 0.0000 0.0230 0.0005
MXNDArraySetGradState 8670 4.4150 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1734 0.7380 0.0000 0.0010 0.0004
MXAutogradBackwardEx 867 479.8530 0.5250 0.9450 0.5535
MXNDArrayGetContext 13698 6.0210 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28797 741.4190 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3468 8477.8701 0.7010 3.7500 2.4446
FullyConnected 5922 571.8420 0.0080 0.3430 0.0966
argmax 1974 39.1400 0.0050 0.0480 0.0198
_backward_pick 1734 27.5370 0.0130 0.0320 0.0159
_backward_mul_scalar 1734 7.5820 0.0030 0.0220 0.0044
DeleteVariable 73680 523.6570 0.0010 0.9010 0.0071
Flatten 1974 82.4020 0.0080 0.1120 0.0417
Activation 7896 367.3220 0.0030 1.9440 0.0465
SetValueOp 1734 11.2670 0.0050 0.0180 0.0065
pick 1734 13.6470 0.0060 0.0280 0.0079
Pooling 3948 5488.3140 0.0640 3.1720 1.3902
Convolution 3948 22415.4258 0.2260 13.2050 5.6777
CopyCPU2CPU 1974 11.2340 0.0030 0.0360 0.0057
log_softmax 1734 54.1000 0.0150 0.0550 0.0312
_backward_copy 1734 89.8370 0.0220 0.0760 0.0518
_backward_Convolution 3468 32070.5586 2.6240 12.5610 9.2476
_mul_scalar 1734 6.1860 0.0020 0.0230 0.0036
_backward_log_softmax 1734 29.6290 0.0090 0.0400 0.0171
mean 5442 104.3210 0.0080 0.0730 0.0192
_backward_mean 1734 19.0440 0.0080 0.1030 0.0110
_backward_FullyConnected 5202 998.2730 0.0280 0.5940 0.1919
WaitForVar 3708 32.9870 0.0060 0.0280 0.0089
broadcast_equal 1974 9.8650 0.0030 0.0110 0.0050
sgd_update 17340 173.6720 0.0020 0.0790 0.0100
_backward_Activation 6936 512.8820 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67832 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5928 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 867 3449.8579 0.3060 73.5870 3.9791
MXNDArrayCreateEx 988 34.2280 0.0310 0.0640 0.0346
MXNDArrayWaitAll 868 18842.5215 5.6130 25.0610 21.7080
MXNDArraySyncCopyToCPU 1856 14551.2744 0.0600 24.1980 7.8401
MXNDArrayGetDType 1856 0.6770 0.0000 0.0010 0.0004
MXNDArrayGetShape 4700 2.1890 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1976 17.3910 0.0070 0.0280 0.0088
MXNDArrayFree 22124 227.3190 0.0000 0.9140 0.0103
MXNet C API Calls 99476 586.8330 487.3580 586.8330 49.7375
MXNet C API Concurrency 198951 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1736 0.8270 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8680 4.4860 0.0000 0.0230 0.0005
MXNDArraySetGradState 8680 4.4220 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1736 0.7390 0.0000 0.0010 0.0004
MXAutogradBackwardEx 868 480.3980 0.5250 0.9450 0.5535
MXNDArrayGetContext 13712 6.0260 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28828 742.1880 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3472 8487.6064 0.7010 3.7500 2.4446
FullyConnected 5928 572.6480 0.0080 0.3430 0.0966
argmax 1976 39.1840 0.0050 0.0480 0.0198
_backward_pick 1736 27.5700 0.0130 0.0320 0.0159
_backward_mul_scalar 1736 7.5900 0.0030 0.0220 0.0044
DeleteVariable 73760 524.1780 0.0010 0.9010 0.0071
Flatten 1976 82.4800 0.0080 0.1120 0.0417
Activation 7904 367.7030 0.0030 1.9440 0.0465
SetValueOp 1736 11.2800 0.0050 0.0180 0.0065
pick 1736 13.6630 0.0060 0.0280 0.0079
Pooling 3952 5493.6660 0.0640 3.1720 1.3901
Convolution 3952 22437.6660 0.2260 13.2050 5.6775
CopyCPU2CPU 1976 11.2430 0.0030 0.0360 0.0057
log_softmax 1736 54.1720 0.0150 0.0550 0.0312
_backward_copy 1736 89.9370 0.0220 0.0760 0.0518
_backward_Convolution 3472 32107.3320 2.6240 12.5610 9.2475
_mul_scalar 1736 6.1930 0.0020 0.0230 0.0036
_backward_log_softmax 1736 29.6720 0.0090 0.0400 0.0171
mean 5448 104.4270 0.0080 0.0730 0.0192
_backward_mean 1736 19.0660 0.0080 0.1030 0.0110
_backward_FullyConnected 5208 999.6720 0.0280 0.5940 0.1919
WaitForVar 3712 33.0220 0.0060 0.0280 0.0089
broadcast_equal 1976 9.8750 0.0030 0.0110 0.0050
sgd_update 17360 173.9490 0.0020 0.0790 0.0100
_backward_Activation 6944 513.4670 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67906 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5934 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 868 3453.7581 0.3060 73.5870 3.9790
MXNDArrayCreateEx 989 34.2610 0.0310 0.0640 0.0346
MXNDArrayWaitAll 869 18864.3867 5.6130 25.0610 21.7082
MXNDArraySyncCopyToCPU 1858 14565.6670 0.0600 24.1980 7.8394
MXNDArrayGetDType 1858 0.6770 0.0000 0.0010 0.0004
MXNDArrayGetShape 4705 2.1930 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1978 17.4090 0.0070 0.0280 0.0088
MXNDArrayFree 22147 227.5490 0.0000 0.9140 0.0103
MXNet C API Calls 99583 586.9400 487.3580 586.9400 49.7910
MXNet C API Concurrency 199165 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1738 0.8280 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8690 4.4920 0.0000 0.0230 0.0005
MXNDArraySetGradState 8690 4.4250 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1738 0.7400 0.0000 0.0010 0.0004
MXAutogradBackwardEx 869 480.9290 0.5250 0.9450 0.5534
MXNDArrayGetContext 13726 6.0350 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28859 742.9650 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3476 8497.3848 0.7010 3.7500 2.4446
FullyConnected 5934 573.1780 0.0080 0.3430 0.0966
argmax 1978 39.2210 0.0050 0.0480 0.0198
_backward_pick 1738 27.6010 0.0130 0.0320 0.0159
_backward_mul_scalar 1738 7.5990 0.0030 0.0220 0.0044
DeleteVariable 73840 524.6930 0.0010 0.9010 0.0071
Flatten 1978 82.5590 0.0080 0.1120 0.0417
Activation 7912 368.0550 0.0030 1.9440 0.0465
SetValueOp 1738 11.2930 0.0050 0.0180 0.0065
pick 1738 13.6790 0.0060 0.0280 0.0079
Pooling 3956 5499.0479 0.0640 3.1720 1.3901
Convolution 3956 22459.8066 0.2260 13.2050 5.6774
CopyCPU2CPU 1978 11.2530 0.0030 0.0360 0.0057
log_softmax 1738 54.2310 0.0150 0.0550 0.0312
_backward_copy 1738 90.0410 0.0220 0.0760 0.0518
_backward_Convolution 3476 32144.3652 2.6240 12.5610 9.2475
_mul_scalar 1738 6.2000 0.0020 0.0230 0.0036
_backward_log_softmax 1738 29.7040 0.0090 0.0400 0.0171
mean 5454 104.5370 0.0080 0.0730 0.0192
_backward_mean 1738 19.0870 0.0080 0.1030 0.0110
_backward_FullyConnected 5214 1000.7410 0.0280 0.5940 0.1919
WaitForVar 3716 33.0580 0.0060 0.0280 0.0089
broadcast_equal 1978 9.8850 0.0030 0.0110 0.0050
sgd_update 17380 174.1030 0.0020 0.0790 0.0100
_backward_Activation 6952 514.0930 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 67980 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5940 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 869 3457.6560 0.3060 73.5870 3.9789
MXNDArrayCreateEx 990 34.2940 0.0310 0.0640 0.0346
MXNDArrayWaitAll 870 18886.2383 5.6130 25.0610 21.7083
MXNDArraySyncCopyToCPU 1860 14580.1074 0.0600 24.1980 7.8388
MXNDArrayGetDType 1860 0.6780 0.0000 0.0010 0.0004
MXNDArrayGetShape 4710 2.1940 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1980 17.4250 0.0070 0.0280 0.0088
MXNDArrayFree 22170 227.7690 0.0000 0.9140 0.0103
MXNet C API Calls 99690 587.0470 487.3580 587.0470 49.8445
MXNet C API Concurrency 199379 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1740 0.8290 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8700 4.4960 0.0000 0.0230 0.0005
MXNDArraySetGradState 8700 4.4300 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1740 0.7410 0.0000 0.0010 0.0004
MXAutogradBackwardEx 870 481.4590 0.5250 0.9450 0.5534
MXNDArrayGetContext 13740 6.0430 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28890 743.7350 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3480 8506.9707 0.7010 3.7500 2.4445
FullyConnected 5940 573.7220 0.0080 0.3430 0.0966
argmax 1980 39.2590 0.0050 0.0480 0.0198
_backward_pick 1740 27.6340 0.0130 0.0320 0.0159
_backward_mul_scalar 1740 7.6070 0.0030 0.0220 0.0044
DeleteVariable 73920 525.2040 0.0010 0.9010 0.0071
Flatten 1980 82.6390 0.0080 0.1120 0.0417
Activation 7920 368.4070 0.0030 1.9440 0.0465
SetValueOp 1740 11.3070 0.0050 0.0180 0.0065
pick 1740 13.6940 0.0060 0.0280 0.0079
Pooling 3960 5504.3828 0.0640 3.1720 1.3900
Convolution 3960 22482.0547 0.2260 13.2050 5.6773
CopyCPU2CPU 1980 11.2630 0.0030 0.0360 0.0057
log_softmax 1740 54.2900 0.0150 0.0550 0.0312
_backward_copy 1740 90.1450 0.0220 0.0760 0.0518
_backward_Convolution 3480 32181.4941 2.6240 12.5610 9.2476
_mul_scalar 1740 6.2070 0.0020 0.0230 0.0036
_backward_log_softmax 1740 29.7370 0.0090 0.0400 0.0171
mean 5460 104.6520 0.0080 0.0730 0.0192
_backward_mean 1740 19.1080 0.0080 0.1030 0.0110
_backward_FullyConnected 5220 1001.8560 0.0280 0.5940 0.1919
WaitForVar 3720 33.0920 0.0060 0.0280 0.0089
broadcast_equal 1980 9.8940 0.0030 0.0110 0.0050
sgd_update 17400 174.3080 0.0020 0.0790 0.0100
_backward_Activation 6960 514.6530 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68054 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5946 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 870 3461.5391 0.3060 73.5870 3.9788
MXNDArrayCreateEx 991 34.3270 0.0310 0.0640 0.0346
MXNDArrayWaitAll 871 18907.9531 5.6130 25.0610 21.7083
MXNDArraySyncCopyToCPU 1862 14594.3848 0.0600 24.1980 7.8380
MXNDArrayGetDType 1862 0.6780 0.0000 0.0010 0.0004
MXNDArrayGetShape 4715 2.1960 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1982 17.4420 0.0070 0.0280 0.0088
MXNDArrayFree 22193 227.9950 0.0000 0.9140 0.0103
MXNet C API Calls 99797 587.1540 487.3580 587.1540 49.8980
MXNet C API Concurrency 199593 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1742 0.8300 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8710 4.5020 0.0000 0.0230 0.0005
MXNDArraySetGradState 8710 4.4360 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1742 0.7430 0.0000 0.0010 0.0004
MXAutogradBackwardEx 871 482.0150 0.5250 0.9450 0.5534
MXNDArrayGetContext 13754 6.0470 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28921 744.5520 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3484 8516.6699 0.7010 3.7500 2.4445
FullyConnected 5946 574.2450 0.0080 0.3430 0.0966
argmax 1982 39.2980 0.0050 0.0480 0.0198
_backward_pick 1742 27.6630 0.0130 0.0320 0.0159
_backward_mul_scalar 1742 7.6160 0.0030 0.0220 0.0044
DeleteVariable 74000 525.7430 0.0010 0.9010 0.0071
Flatten 1982 82.7180 0.0080 0.1120 0.0417
Activation 7928 368.7630 0.0030 1.9440 0.0465
SetValueOp 1742 11.3190 0.0050 0.0180 0.0065
pick 1742 13.7080 0.0060 0.0280 0.0079
Pooling 3964 5509.8188 0.0640 3.1720 1.3900
Convolution 3964 22504.2344 0.2260 13.2050 5.6772
CopyCPU2CPU 1982 11.2720 0.0030 0.0360 0.0057
log_softmax 1742 54.3500 0.0150 0.0550 0.0312
_backward_copy 1742 90.2490 0.0220 0.0760 0.0518
_backward_Convolution 3484 32218.2871 2.6240 12.5610 9.2475
_mul_scalar 1742 6.2130 0.0020 0.0230 0.0036
_backward_log_softmax 1742 29.7680 0.0090 0.0400 0.0171
mean 5466 104.8050 0.0080 0.0730 0.0192
_backward_mean 1742 19.1290 0.0080 0.1030 0.0110
_backward_FullyConnected 5226 1002.9210 0.0280 0.5940 0.1919
WaitForVar 3724 33.1270 0.0060 0.0280 0.0089
broadcast_equal 1982 9.9040 0.0030 0.0110 0.0050
sgd_update 17420 174.5110 0.0020 0.0790 0.0100
_backward_Activation 6968 515.1810 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68128 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5952 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 871 3465.4290 0.3060 73.5870 3.9787
MXNDArrayCreateEx 992 34.3600 0.0310 0.0640 0.0346
MXNDArrayWaitAll 872 18929.6504 5.6130 25.0610 21.7083
MXNDArraySyncCopyToCPU 1864 14609.1514 0.0600 24.1980 7.8375
MXNDArrayGetDType 1864 0.6780 0.0000 0.0010 0.0004
MXNDArrayGetShape 4720 2.1970 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1984 17.4590 0.0070 0.0280 0.0088
MXNDArrayFree 22216 228.2430 0.0000 0.9140 0.0103
MXNet C API Calls 99904 587.2610 487.3580 587.2610 49.9515
MXNet C API Concurrency 199807 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1744 0.8310 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8720 4.5070 0.0000 0.0230 0.0005
MXNDArraySetGradState 8720 4.4430 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1744 0.7450 0.0000 0.0010 0.0004
MXAutogradBackwardEx 872 482.5460 0.5250 0.9450 0.5534
MXNDArrayGetContext 13768 6.0520 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28952 745.3200 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3488 8526.4404 0.7010 3.7500 2.4445
FullyConnected 5952 574.9680 0.0080 0.3430 0.0966
argmax 1984 39.3420 0.0050 0.0480 0.0198
_backward_pick 1744 27.6960 0.0130 0.0320 0.0159
_backward_mul_scalar 1744 7.6250 0.0030 0.0220 0.0044
DeleteVariable 74080 526.2930 0.0010 0.9010 0.0071
Flatten 1984 82.7960 0.0080 0.1120 0.0417
Activation 7936 369.1480 0.0030 1.9440 0.0465
SetValueOp 1744 11.3320 0.0050 0.0180 0.0065
pick 1744 13.7240 0.0060 0.0280 0.0079
Pooling 3968 5515.1748 0.0640 3.1720 1.3899
Convolution 3968 22526.4023 0.2260 13.2050 5.6770
CopyCPU2CPU 1984 11.2820 0.0030 0.0360 0.0057
log_softmax 1744 54.4220 0.0150 0.0550 0.0312
_backward_copy 1744 90.3500 0.0220 0.0760 0.0518
_backward_Convolution 3488 32254.9961 2.6240 12.5610 9.2474
_mul_scalar 1744 6.2210 0.0020 0.0230 0.0036
_backward_log_softmax 1744 29.8090 0.0090 0.0400 0.0171
mean 5472 104.9090 0.0080 0.0730 0.0192
_backward_mean 1744 19.1510 0.0080 0.1030 0.0110
_backward_FullyConnected 5232 1004.3330 0.0280 0.5940 0.1920
WaitForVar 3728 33.1610 0.0060 0.0280 0.0089
broadcast_equal 1984 9.9130 0.0030 0.0110 0.0050
sgd_update 17440 174.8180 0.0020 0.0790 0.0100
_backward_Activation 6976 515.7550 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68202 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5958 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 872 3469.3220 0.3060 73.5870 3.9786
MXNDArrayCreateEx 993 34.3920 0.0310 0.0640 0.0346
MXNDArrayWaitAll 873 18951.5176 5.6130 25.0610 21.7085
MXNDArraySyncCopyToCPU 1866 14623.5322 0.0600 24.1980 7.8368
MXNDArrayGetDType 1866 0.6780 0.0000 0.0010 0.0004
MXNDArrayGetShape 4725 2.2000 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1986 17.4760 0.0070 0.0280 0.0088
MXNDArrayFree 22239 228.4660 0.0000 0.9140 0.0103
MXNet C API Calls 100011 587.3680 487.3580 587.3680 50.0050
MXNet C API Concurrency 200021 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1746 0.8310 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8730 4.5090 0.0000 0.0230 0.0005
MXNDArraySetGradState 8730 4.4450 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1746 0.7470 0.0000 0.0010 0.0004
MXAutogradBackwardEx 873 483.0760 0.5250 0.9450 0.5534
MXNDArrayGetContext 13782 6.0600 0.0000 0.0190 0.0004
MXImperativeInvokeEx 28983 746.0850 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3492 8536.1621 0.7010 3.7500 2.4445
FullyConnected 5958 575.5140 0.0080 0.3430 0.0966
argmax 1986 39.3810 0.0050 0.0480 0.0198
_backward_pick 1746 27.7270 0.0130 0.0320 0.0159
_backward_mul_scalar 1746 7.6340 0.0030 0.0220 0.0044
DeleteVariable 74160 526.8220 0.0010 0.9010 0.0071
Flatten 1986 82.8760 0.0080 0.1120 0.0417
Activation 7944 369.4990 0.0030 1.9440 0.0465
SetValueOp 1746 11.3440 0.0050 0.0180 0.0065
pick 1746 13.7390 0.0060 0.0280 0.0079
Pooling 3972 5520.5459 0.0640 3.1720 1.3899
Convolution 3972 22548.5547 0.2260 13.2050 5.6769
CopyCPU2CPU 1986 11.2920 0.0030 0.0360 0.0057
log_softmax 1746 54.4820 0.0150 0.0550 0.0312
_backward_copy 1746 90.4540 0.0220 0.0760 0.0518
_backward_Convolution 3492 32292.0645 2.6240 12.5610 9.2474
_mul_scalar 1746 6.2270 0.0020 0.0230 0.0036
_backward_log_softmax 1746 29.8410 0.0090 0.0400 0.0171
mean 5478 105.0280 0.0080 0.0730 0.0192
_backward_mean 1746 19.1720 0.0080 0.1030 0.0110
_backward_FullyConnected 5238 1005.3990 0.0280 0.5940 0.1919
WaitForVar 3732 33.1980 0.0060 0.0280 0.0089
broadcast_equal 1986 9.9220 0.0030 0.0110 0.0050
sgd_update 17460 175.0170 0.0020 0.0790 0.0100
_backward_Activation 6984 516.3020 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68276 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5964 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 873 3473.2339 0.3060 73.5870 3.9785
MXNDArrayCreateEx 994 34.4250 0.0310 0.0640 0.0346
MXNDArrayWaitAll 874 18973.3887 5.6130 25.0610 21.7087
MXNDArraySyncCopyToCPU 1868 14638.0059 0.0600 24.1980 7.8362
MXNDArrayGetDType 1868 0.6780 0.0000 0.0010 0.0004
MXNDArrayGetShape 4730 2.2000 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1988 17.4930 0.0070 0.0280 0.0088
MXNDArrayFree 22262 228.6990 0.0000 0.9140 0.0103
MXNet C API Calls 100118 587.4750 487.3580 587.4750 50.0585
MXNet C API Concurrency 200235 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1748 0.8320 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8740 4.5150 0.0000 0.0230 0.0005
MXNDArraySetGradState 8740 4.4520 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1748 0.7480 0.0000 0.0010 0.0004
MXAutogradBackwardEx 874 483.6090 0.5250 0.9450 0.5533
MXNDArrayGetContext 13796 6.0680 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29014 746.8830 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3496 8545.9707 0.7010 3.7500 2.4445
FullyConnected 5964 576.0580 0.0080 0.3430 0.0966
argmax 1988 39.4180 0.0050 0.0480 0.0198
_backward_pick 1748 27.7580 0.0130 0.0320 0.0159
_backward_mul_scalar 1748 7.6420 0.0030 0.0220 0.0044
DeleteVariable 74240 527.3640 0.0010 0.9010 0.0071
Flatten 1988 82.9570 0.0080 0.1120 0.0417
Activation 7952 369.8620 0.0030 1.9440 0.0465
SetValueOp 1748 11.3580 0.0050 0.0180 0.0065
pick 1748 13.7550 0.0060 0.0280 0.0079
Pooling 3976 5525.9131 0.0640 3.1720 1.3898
Convolution 3976 22570.7734 0.2260 13.2050 5.6768
CopyCPU2CPU 1988 11.3020 0.0030 0.0360 0.0057
log_softmax 1748 54.5410 0.0150 0.0550 0.0312
_backward_copy 1748 90.5580 0.0220 0.0760 0.0518
_backward_Convolution 3496 32329.1152 2.6240 12.5610 9.2475
_mul_scalar 1748 6.2340 0.0020 0.0230 0.0036
_backward_log_softmax 1748 29.8740 0.0090 0.0400 0.0171
mean 5484 105.1450 0.0080 0.0730 0.0192
_backward_mean 1748 19.1930 0.0080 0.1030 0.0110
_backward_FullyConnected 5244 1006.4630 0.0280 0.5940 0.1919
WaitForVar 3736 33.2350 0.0060 0.0280 0.0089
broadcast_equal 1988 9.9320 0.0030 0.0110 0.0050
sgd_update 17480 175.1690 0.0020 0.0790 0.0100
_backward_Activation 6992 516.9290 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68350 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5970 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 874 3477.1321 0.3060 73.5870 3.9784
MXNDArrayCreateEx 995 34.4580 0.0310 0.0640 0.0346
MXNDArrayWaitAll 875 18995.1113 5.6130 25.0610 21.7087
MXNDArraySyncCopyToCPU 1870 14652.1768 0.0600 24.1980 7.8354
MXNDArrayGetDType 1870 0.6790 0.0000 0.0010 0.0004
MXNDArrayGetShape 4735 2.2020 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1990 17.5100 0.0070 0.0280 0.0088
MXNDArrayFree 22285 228.9340 0.0000 0.9140 0.0103
MXNet C API Calls 100225 587.5820 487.3580 587.5820 50.1120
MXNet C API Concurrency 200449 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1750 0.8320 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8750 4.5210 0.0000 0.0230 0.0005
MXNDArraySetGradState 8750 4.4570 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1750 0.7480 0.0000 0.0010 0.0004
MXAutogradBackwardEx 875 484.1380 0.5250 0.9450 0.5533
MXNDArrayGetContext 13810 6.0750 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29045 747.6750 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3500 8555.6982 0.7010 3.7500 2.4445
FullyConnected 5970 576.6250 0.0080 0.3430 0.0966
argmax 1990 39.4560 0.0050 0.0480 0.0198
_backward_pick 1750 27.7900 0.0130 0.0320 0.0159
_backward_mul_scalar 1750 7.6510 0.0030 0.0220 0.0044
DeleteVariable 74320 527.9000 0.0010 0.9010 0.0071
Flatten 1990 83.0360 0.0080 0.1120 0.0417
Activation 7960 370.2150 0.0030 1.9440 0.0465
SetValueOp 1750 11.3710 0.0050 0.0180 0.0065
pick 1750 13.7700 0.0060 0.0280 0.0079
Pooling 3980 5531.2529 0.0640 3.1720 1.3898
Convolution 3980 22592.9551 0.2260 13.2050 5.6766
CopyCPU2CPU 1990 11.3110 0.0030 0.0360 0.0057
log_softmax 1750 54.6000 0.0150 0.0550 0.0312
_backward_copy 1750 90.6620 0.0220 0.0760 0.0518
_backward_Convolution 3500 32365.8965 2.6240 12.5610 9.2474
_mul_scalar 1750 6.2410 0.0020 0.0230 0.0036
_backward_log_softmax 1750 29.9070 0.0090 0.0400 0.0171
mean 5490 105.2620 0.0080 0.0730 0.0192
_backward_mean 1750 19.2140 0.0080 0.1030 0.0110
_backward_FullyConnected 5250 1007.5280 0.0280 0.5940 0.1919
WaitForVar 3740 33.2750 0.0060 0.0280 0.0089
broadcast_equal 1990 9.9420 0.0030 0.0110 0.0050
sgd_update 17500 175.3680 0.0020 0.0790 0.0100
_backward_Activation 7000 517.4660 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68424 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5976 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 875 3481.0330 0.3060 73.5870 3.9783
MXNDArrayCreateEx 996 34.4910 0.0310 0.0640 0.0346
MXNDArrayWaitAll 876 19016.7500 5.6130 25.0610 21.7086
MXNDArraySyncCopyToCPU 1872 14666.7930 0.0600 24.1980 7.8348
MXNDArrayGetDType 1872 0.6800 0.0000 0.0010 0.0004
MXNDArrayGetShape 4740 2.2040 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1992 17.5270 0.0070 0.0280 0.0088
MXNDArrayFree 22308 229.1630 0.0000 0.9140 0.0103
MXNet C API Calls 100332 587.6890 487.3580 587.6890 50.1655
MXNet C API Concurrency 200663 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1752 0.8340 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8760 4.5260 0.0000 0.0230 0.0005
MXNDArraySetGradState 8760 4.4640 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1752 0.7480 0.0000 0.0010 0.0004
MXAutogradBackwardEx 876 484.6770 0.5250 0.9450 0.5533
MXNDArrayGetContext 13824 6.0800 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29076 748.4620 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3504 8565.2676 0.7010 3.7500 2.4444
FullyConnected 5976 577.3790 0.0080 0.3430 0.0966
argmax 1992 39.5000 0.0050 0.0480 0.0198
_backward_pick 1752 27.8240 0.0130 0.0320 0.0159
_backward_mul_scalar 1752 7.6590 0.0030 0.0220 0.0044
DeleteVariable 74400 528.4360 0.0010 0.9010 0.0071
Flatten 1992 83.1130 0.0080 0.1120 0.0417
Activation 7968 370.5940 0.0030 1.9440 0.0465
SetValueOp 1752 11.3850 0.0050 0.0180 0.0065
pick 1752 13.7860 0.0060 0.0280 0.0079
Pooling 3984 5536.6040 0.0640 3.1720 1.3897
Convolution 3984 22615.1348 0.2260 13.2050 5.6765
CopyCPU2CPU 1992 11.3210 0.0030 0.0360 0.0057
log_softmax 1752 54.6730 0.0150 0.0550 0.0312
_backward_copy 1752 90.7620 0.0220 0.0760 0.0518
_backward_Convolution 3504 32402.6855 2.6240 12.5610 9.2473
_mul_scalar 1752 6.2480 0.0020 0.0230 0.0036
_backward_log_softmax 1752 29.9490 0.0090 0.0400 0.0171
mean 5496 105.3660 0.0080 0.0730 0.0192
_backward_mean 1752 19.2360 0.0080 0.1030 0.0110
_backward_FullyConnected 5256 1008.9390 0.0280 0.5940 0.1920
WaitForVar 3744 33.3100 0.0060 0.0280 0.0089
broadcast_equal 1992 9.9520 0.0030 0.0110 0.0050
sgd_update 17520 175.6430 0.0020 0.0790 0.0100
_backward_Activation 7008 518.0520 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68498 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5982 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 876 3484.9451 0.3060 73.5870 3.9782
MXNDArrayCreateEx 997 34.5250 0.0310 0.0640 0.0346
MXNDArrayWaitAll 877 19038.5391 5.6130 25.0610 21.7087
MXNDArraySyncCopyToCPU 1874 14681.1758 0.0600 24.1980 7.8341
MXNDArrayGetDType 1874 0.6810 0.0000 0.0010 0.0004
MXNDArrayGetShape 4745 2.2060 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1994 17.5440 0.0070 0.0280 0.0088
MXNDArrayFree 22331 229.3860 0.0000 0.9140 0.0103
MXNet C API Calls 100439 587.7960 487.3580 587.7960 50.2190
MXNet C API Concurrency 200877 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1754 0.8350 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8770 4.5320 0.0000 0.0230 0.0005
MXNDArraySetGradState 8770 4.4680 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1754 0.7480 0.0000 0.0010 0.0004
MXAutogradBackwardEx 877 485.2150 0.5250 0.9450 0.5533
MXNDArrayGetContext 13838 6.0840 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29107 749.2340 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3508 8574.8604 0.7010 3.7500 2.4444
FullyConnected 5982 577.9050 0.0080 0.3430 0.0966
argmax 1994 39.5400 0.0050 0.0480 0.0198
_backward_pick 1754 27.8580 0.0130 0.0320 0.0159
_backward_mul_scalar 1754 7.6670 0.0030 0.0220 0.0044
DeleteVariable 74480 528.9660 0.0010 0.9010 0.0071
Flatten 1994 83.1960 0.0080 0.1120 0.0417
Activation 7976 370.9460 0.0030 1.9440 0.0465
SetValueOp 1754 11.3980 0.0050 0.0180 0.0065
pick 1754 13.8010 0.0060 0.0280 0.0079
Pooling 3988 5541.9238 0.0640 3.1720 1.3896
Convolution 3988 22637.2812 0.2260 13.2050 5.6763
CopyCPU2CPU 1994 11.3310 0.0030 0.0360 0.0057
log_softmax 1754 54.7320 0.0150 0.0550 0.0312
_backward_copy 1754 90.8660 0.0220 0.0760 0.0518
_backward_Convolution 3508 32439.7480 2.6240 12.5610 9.2474
_mul_scalar 1754 6.2550 0.0020 0.0230 0.0036
_backward_log_softmax 1754 29.9810 0.0090 0.0400 0.0171
mean 5502 105.4880 0.0080 0.0730 0.0192
_backward_mean 1754 19.2570 0.0080 0.1030 0.0110
_backward_FullyConnected 5262 1010.0580 0.0280 0.5940 0.1920
WaitForVar 3748 33.3440 0.0060 0.0280 0.0089
broadcast_equal 1994 9.9620 0.0030 0.0110 0.0050
sgd_update 17540 175.8410 0.0020 0.0790 0.0100
_backward_Activation 7016 518.5990 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68572 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5988 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 877 3488.8931 0.3060 73.5870 3.9782
MXNDArrayCreateEx 998 34.5620 0.0310 0.0640 0.0346
MXNDArrayWaitAll 878 19050.3281 5.6130 25.0610 21.6974
MXNDArraySyncCopyToCPU 1876 14705.1602 0.0600 24.1980 7.8386
MXNDArrayGetDType 1876 0.6810 0.0000 0.0010 0.0004
MXNDArrayGetShape 4750 2.2100 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1996 17.5610 0.0070 0.0280 0.0088
MXNDArrayFree 22354 229.6310 0.0000 0.9140 0.0103
MXNet C API Calls 100546 587.9030 487.3580 587.9030 50.2725
MXNet C API Concurrency 201091 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1756 0.8370 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8780 4.5370 0.0000 0.0230 0.0005
MXNDArraySetGradState 8780 4.4740 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1756 0.7480 0.0000 0.0010 0.0004
MXAutogradBackwardEx 878 485.7580 0.5250 0.9450 0.5533
MXNDArrayGetContext 13852 6.0910 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29138 750.0500 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3512 8584.5684 0.7010 3.7500 2.4444
FullyConnected 5988 578.4400 0.0080 0.3430 0.0966
argmax 1996 39.5790 0.0050 0.0480 0.0198
_backward_pick 1756 27.8890 0.0130 0.0320 0.0159
_backward_mul_scalar 1756 7.6760 0.0030 0.0220 0.0044
DeleteVariable 74560 529.5430 0.0010 0.9010 0.0071
Flatten 1996 83.2870 0.0080 0.1120 0.0417
Activation 7984 371.3060 0.0030 1.9440 0.0465
SetValueOp 1756 11.4110 0.0050 0.0180 0.0065
pick 1756 13.8170 0.0060 0.0280 0.0079
Pooling 3992 5547.3120 0.0640 3.1720 1.3896
Convolution 3992 22659.5781 0.2260 13.2050 5.6762
CopyCPU2CPU 1996 11.3930 0.0030 0.0360 0.0057
log_softmax 1756 54.7920 0.0150 0.0550 0.0312
_backward_copy 1756 90.9720 0.0220 0.0760 0.0518
_backward_Convolution 3512 32476.8379 2.6240 12.5610 9.2474
_mul_scalar 1756 6.2630 0.0020 0.0230 0.0036
_backward_log_softmax 1756 30.0130 0.0090 0.0400 0.0171
mean 5508 105.5660 0.0080 0.0730 0.0192
_backward_mean 1756 19.2780 0.0080 0.1030 0.0110
_backward_FullyConnected 5268 1011.1380 0.0280 0.5940 0.1919
WaitForVar 3752 33.3760 0.0060 0.0280 0.0089
broadcast_equal 1996 9.9720 0.0030 0.0110 0.0050
sgd_update 17560 176.0460 0.0020 0.0790 0.0100
_backward_Activation 7024 519.1590 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68646 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 5994 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 878 3492.8501 0.3060 73.5870 3.9782
MXNDArrayCreateEx 999 34.5970 0.0310 0.0640 0.0346
MXNDArrayWaitAll 879 19072.1289 5.6130 25.0610 21.6975
MXNDArraySyncCopyToCPU 1878 14719.4512 0.0600 24.1980 7.8378
MXNDArrayGetDType 1878 0.6810 0.0000 0.0010 0.0004
MXNDArrayGetShape 4755 2.2110 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 1998 17.5780 0.0070 0.0280 0.0088
MXNDArrayFree 22377 229.8760 0.0000 0.9140 0.0103
MXNet C API Calls 100653 588.0100 487.3580 588.0100 50.3260
MXNet C API Concurrency 201305 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1758 0.8380 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8790 4.5420 0.0000 0.0230 0.0005
MXNDArraySetGradState 8790 4.4790 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1758 0.7490 0.0000 0.0010 0.0004
MXAutogradBackwardEx 879 486.3290 0.5250 0.9450 0.5533
MXNDArrayGetContext 13866 6.0980 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29169 750.8770 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3516 8594.3613 0.7010 3.7500 2.4444
FullyConnected 5994 578.9840 0.0080 0.3430 0.0966
argmax 1998 39.6190 0.0050 0.0480 0.0198
_backward_pick 1758 27.9210 0.0130 0.0320 0.0159
_backward_mul_scalar 1758 7.6850 0.0030 0.0220 0.0044
DeleteVariable 74640 530.0970 0.0010 0.9010 0.0071
Flatten 1998 83.3920 0.0080 0.1120 0.0417
Activation 7992 371.6740 0.0030 1.9440 0.0465
SetValueOp 1758 11.4260 0.0050 0.0180 0.0065
pick 1758 13.8340 0.0060 0.0280 0.0079
Pooling 3996 5552.6641 0.0640 3.1720 1.3896
Convolution 3996 22681.8945 0.2260 13.2050 5.6761
CopyCPU2CPU 1998 11.4040 0.0030 0.0360 0.0057
log_softmax 1758 54.8520 0.0150 0.0550 0.0312
_backward_copy 1758 91.0750 0.0220 0.0760 0.0518
_backward_Convolution 3516 32513.7246 2.6240 12.5610 9.2474
_mul_scalar 1758 6.2700 0.0020 0.0230 0.0036
_backward_log_softmax 1758 30.0450 0.0090 0.0400 0.0171
mean 5514 105.6860 0.0080 0.0730 0.0192
_backward_mean 1758 19.3020 0.0080 0.1030 0.0110
_backward_FullyConnected 5274 1012.2680 0.0280 0.5940 0.1919
WaitForVar 3756 33.4110 0.0060 0.0280 0.0089
broadcast_equal 1998 9.9830 0.0030 0.0110 0.0050
sgd_update 17580 176.1960 0.0020 0.0790 0.0100
_backward_Activation 7032 519.7780 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68720 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6000 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 879 3496.8101 0.3060 73.5870 3.9782
MXNDArrayCreateEx 1000 34.6310 0.0310 0.0640 0.0346
MXNDArrayWaitAll 880 19093.7988 5.6130 25.0610 21.6975
MXNDArraySyncCopyToCPU 1880 14734.1924 0.0600 24.1980 7.8373
MXNDArrayGetDType 1880 0.6820 0.0000 0.0010 0.0004
MXNDArrayGetShape 4760 2.2140 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2000 17.5950 0.0070 0.0280 0.0088
MXNDArrayFree 22400 230.1430 0.0000 0.9140 0.0103
MXNet C API Calls 100760 588.1170 487.3580 588.1170 50.3795
MXNet C API Concurrency 201519 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1760 0.8380 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8800 4.5470 0.0000 0.0230 0.0005
MXNDArraySetGradState 8800 4.4860 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1760 0.7500 0.0000 0.0010 0.0004
MXAutogradBackwardEx 880 486.8850 0.5250 0.9450 0.5533
MXNDArrayGetContext 13880 6.1050 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29200 751.6610 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3520 8604.0244 0.7010 3.7500 2.4443
FullyConnected 6000 579.7280 0.0080 0.3430 0.0966
argmax 2000 39.6640 0.0050 0.0480 0.0198
_backward_pick 1760 27.9540 0.0130 0.0320 0.0159
_backward_mul_scalar 1760 7.6940 0.0030 0.0220 0.0044
DeleteVariable 74720 530.6670 0.0010 0.9010 0.0071
Flatten 2000 83.4730 0.0080 0.1120 0.0417
Activation 8000 372.0580 0.0030 1.9440 0.0465
SetValueOp 1760 11.4400 0.0050 0.0180 0.0065
pick 1760 13.8500 0.0060 0.0280 0.0079
Pooling 4000 5558.0439 0.0640 3.1720 1.3895
Convolution 4000 22704.1309 0.2260 13.2050 5.6760
CopyCPU2CPU 2000 11.4140 0.0030 0.0360 0.0057
log_softmax 1760 54.9250 0.0150 0.0550 0.0312
_backward_copy 1760 91.1740 0.0220 0.0760 0.0518
_backward_Convolution 3520 32550.4844 2.6240 12.5610 9.2473
_mul_scalar 1760 6.2770 0.0020 0.0230 0.0036
_backward_log_softmax 1760 30.0870 0.0090 0.0400 0.0171
mean 5520 105.7930 0.0080 0.0730 0.0192
_backward_mean 1760 19.3240 0.0080 0.1030 0.0110
_backward_FullyConnected 5280 1013.6820 0.0280 0.5940 0.1920
WaitForVar 3760 33.4460 0.0060 0.0280 0.0089
broadcast_equal 2000 9.9930 0.0030 0.0110 0.0050
sgd_update 17600 176.4720 0.0020 0.0790 0.0100
_backward_Activation 7040 520.3640 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68794 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6006 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 880 3500.7229 0.3060 73.5870 3.9781
MXNDArrayCreateEx 1001 34.6640 0.0310 0.0640 0.0346
MXNDArrayWaitAll 881 19115.6230 5.6130 25.0610 21.6976
MXNDArraySyncCopyToCPU 1882 14748.6357 0.0600 24.1980 7.8367
MXNDArrayGetDType 1882 0.6840 0.0000 0.0010 0.0004
MXNDArrayGetShape 4765 2.2170 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2002 17.6120 0.0070 0.0280 0.0088
MXNDArrayFree 22423 230.3740 0.0000 0.9140 0.0103
MXNet C API Calls 100867 588.2240 487.3580 588.2240 50.4330
MXNet C API Concurrency 201733 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1762 0.8380 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8810 4.5520 0.0000 0.0230 0.0005
MXNDArraySetGradState 8810 4.4910 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1762 0.7510 0.0000 0.0010 0.0004
MXAutogradBackwardEx 881 487.4180 0.5250 0.9450 0.5533
MXNDArrayGetContext 13894 6.1110 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29231 752.4260 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3524 8613.6768 0.7010 3.7500 2.4443
FullyConnected 6006 580.2760 0.0080 0.3430 0.0966
argmax 2002 39.7030 0.0050 0.0480 0.0198
_backward_pick 1762 27.9880 0.0130 0.0320 0.0159
_backward_mul_scalar 1762 7.7020 0.0030 0.0220 0.0044
DeleteVariable 74800 531.2190 0.0010 0.9010 0.0071
Flatten 2002 83.5530 0.0080 0.1120 0.0417
Activation 8008 372.4100 0.0030 1.9440 0.0465
SetValueOp 1762 11.4530 0.0050 0.0180 0.0065
pick 1762 13.8650 0.0060 0.0280 0.0079
Pooling 4004 5563.3618 0.0640 3.1720 1.3895
Convolution 4004 22726.3320 0.2260 13.2050 5.6759
CopyCPU2CPU 2002 11.4230 0.0030 0.0360 0.0057
log_softmax 1762 54.9840 0.0150 0.0550 0.0312
_backward_copy 1762 91.2760 0.0220 0.0760 0.0518
_backward_Convolution 3524 32587.5352 2.6240 12.5610 9.2473
_mul_scalar 1762 6.2850 0.0020 0.0230 0.0036
_backward_log_softmax 1762 30.1190 0.0090 0.0400 0.0171
mean 5526 105.9350 0.0080 0.0730 0.0192
_backward_mean 1762 19.3460 0.0080 0.1030 0.0110
_backward_FullyConnected 5286 1014.7890 0.0280 0.5940 0.1920
WaitForVar 3764 33.4800 0.0060 0.0280 0.0089
broadcast_equal 2002 10.0020 0.0030 0.0110 0.0050
sgd_update 17620 176.6250 0.0020 0.0790 0.0100
_backward_Activation 7048 520.9920 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68868 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6012 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 881 3504.6250 0.3060 73.5870 3.9780
MXNDArrayCreateEx 1002 34.6980 0.0310 0.0640 0.0346
MXNDArrayWaitAll 882 19137.5254 5.6130 25.0610 21.6979
MXNDArraySyncCopyToCPU 1884 14763.0752 0.0600 24.1980 7.8360
MXNDArrayGetDType 1884 0.6850 0.0000 0.0010 0.0004
MXNDArrayGetShape 4770 2.2210 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2004 17.6290 0.0070 0.0280 0.0088
MXNDArrayFree 22446 230.6100 0.0000 0.9140 0.0103
MXNet C API Calls 100974 588.3310 487.3580 588.3310 50.4865
MXNet C API Concurrency 201947 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1764 0.8400 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8820 4.5580 0.0000 0.0230 0.0005
MXNDArraySetGradState 8820 4.4940 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1764 0.7520 0.0000 0.0010 0.0004
MXAutogradBackwardEx 882 487.9750 0.5250 0.9450 0.5533
MXNDArrayGetContext 13908 6.1170 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29262 753.1960 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3528 8623.4365 0.7010 3.7500 2.4443
FullyConnected 6012 580.8230 0.0080 0.3430 0.0966
argmax 2004 39.7420 0.0050 0.0480 0.0198
_backward_pick 1764 28.0190 0.0130 0.0320 0.0159
_backward_mul_scalar 1764 7.7100 0.0030 0.0220 0.0044
DeleteVariable 74880 531.7500 0.0010 0.9010 0.0071
Flatten 2004 83.6320 0.0080 0.1120 0.0417
Activation 8016 372.7610 0.0030 1.9440 0.0465
SetValueOp 1764 11.4650 0.0050 0.0180 0.0065
pick 1764 13.8800 0.0060 0.0280 0.0079
Pooling 4008 5568.7539 0.0640 3.1720 1.3894
Convolution 4008 22748.5859 0.2260 13.2050 5.6758
CopyCPU2CPU 2004 11.4330 0.0030 0.0360 0.0057
log_softmax 1764 55.0440 0.0150 0.0550 0.0312
_backward_copy 1764 91.3790 0.0220 0.0760 0.0518
_backward_Convolution 3528 32624.6445 2.6240 12.5610 9.2473
_mul_scalar 1764 6.2920 0.0020 0.0230 0.0036
_backward_log_softmax 1764 30.1510 0.0090 0.0400 0.0171
mean 5532 106.0470 0.0080 0.0730 0.0192
_backward_mean 1764 19.3670 0.0080 0.1030 0.0110
_backward_FullyConnected 5292 1015.8540 0.0280 0.5940 0.1920
WaitForVar 3768 33.5180 0.0060 0.0280 0.0089
broadcast_equal 2004 10.0120 0.0030 0.0110 0.0050
sgd_update 17640 176.8270 0.0020 0.0790 0.0100
_backward_Activation 7056 521.5420 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 68942 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6018 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 882 3506.1641 0.3060 73.5870 3.9752
MXNDArrayCreateEx 1003 34.7310 0.0310 0.0640 0.0346
MXNDArrayWaitAll 883 19159.2891 5.6130 25.0610 21.6979
MXNDArraySyncCopyToCPU 1886 14777.5156 0.0600 24.1980 7.8354
MXNDArrayGetDType 1886 0.6850 0.0000 0.0010 0.0004
MXNDArrayGetShape 4775 2.2250 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2006 17.6470 0.0070 0.0280 0.0088
MXNDArrayFree 22469 230.8370 0.0000 0.9140 0.0103
MXNet C API Calls 101081 588.4380 487.3580 588.4380 50.5400
MXNet C API Concurrency 202161 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1766 0.8410 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8830 4.5620 0.0000 0.0230 0.0005
MXNDArraySetGradState 8830 4.4970 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1766 0.7540 0.0000 0.0010 0.0004
MXAutogradBackwardEx 883 488.5070 0.5250 0.9450 0.5532
MXNDArrayGetContext 13922 6.1240 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29293 753.9720 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3532 8633.1934 0.7010 3.7500 2.4443
FullyConnected 6018 581.4000 0.0080 0.3430 0.0966
argmax 2006 39.7810 0.0050 0.0480 0.0198
_backward_pick 1766 28.0510 0.0130 0.0320 0.0159
_backward_mul_scalar 1766 7.7190 0.0030 0.0220 0.0044
DeleteVariable 74960 532.2890 0.0010 0.9010 0.0071
Flatten 2006 83.7110 0.0080 0.1120 0.0417
Activation 8024 373.1140 0.0030 1.9440 0.0465
SetValueOp 1766 11.4790 0.0050 0.0180 0.0065
pick 1766 13.8950 0.0060 0.0280 0.0079
Pooling 4012 5574.1201 0.0640 3.1720 1.3894
Convolution 4012 22770.8184 0.2260 13.2050 5.6757
CopyCPU2CPU 2006 11.4420 0.0030 0.0360 0.0057
log_softmax 1766 55.1020 0.0150 0.0550 0.0312
_backward_copy 1766 91.4820 0.0220 0.0760 0.0518
_backward_Convolution 3532 32661.5195 2.6240 12.5610 9.2473
_mul_scalar 1766 6.2990 0.0020 0.0230 0.0036
_backward_log_softmax 1766 30.1820 0.0090 0.0400 0.0171
mean 5538 106.1650 0.0080 0.0730 0.0192
_backward_mean 1766 19.3880 0.0080 0.1030 0.0110
_backward_FullyConnected 5298 1016.9260 0.0280 0.5940 0.1919
WaitForVar 3772 33.5530 0.0060 0.0280 0.0089
broadcast_equal 2006 10.0210 0.0030 0.0110 0.0050
sgd_update 17660 177.0270 0.0020 0.0790 0.0100
_backward_Activation 7064 522.0880 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69016 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6024 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 883 3510.0730 0.3060 73.5870 3.9752
MXNDArrayCreateEx 1004 34.7640 0.0310 0.0640 0.0346
MXNDArrayWaitAll 884 19181.0039 5.6130 25.0610 21.6980
MXNDArraySyncCopyToCPU 1888 14792.3037 0.0600 24.1980 7.8349
MXNDArrayGetDType 1888 0.6870 0.0000 0.0010 0.0004
MXNDArrayGetShape 4780 2.2280 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2008 17.6640 0.0070 0.0280 0.0088
MXNDArrayFree 22492 231.0650 0.0000 0.9140 0.0103
MXNet C API Calls 101188 588.5450 487.3580 588.5450 50.5935
MXNet C API Concurrency 202375 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1768 0.8410 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8840 4.5650 0.0000 0.0230 0.0005
MXNDArraySetGradState 8840 4.5030 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1768 0.7540 0.0000 0.0010 0.0004
MXAutogradBackwardEx 884 489.0450 0.5250 0.9450 0.5532
MXNDArrayGetContext 13936 6.1310 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29324 754.7450 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3536 8642.9551 0.7010 3.7500 2.4443
FullyConnected 6024 582.1290 0.0080 0.3430 0.0966
argmax 2008 39.8250 0.0050 0.0480 0.0198
_backward_pick 1768 28.0830 0.0130 0.0320 0.0159
_backward_mul_scalar 1768 7.7280 0.0030 0.0220 0.0044
DeleteVariable 75040 532.8260 0.0010 0.9010 0.0071
Flatten 2008 83.7900 0.0080 0.1120 0.0417
Activation 8032 373.4950 0.0030 1.9440 0.0465
SetValueOp 1768 11.4920 0.0050 0.0180 0.0065
pick 1768 13.9110 0.0060 0.0280 0.0079
Pooling 4016 5579.5200 0.0640 3.1720 1.3893
Convolution 4016 22793.0684 0.2260 13.2050 5.6756
CopyCPU2CPU 2008 11.4520 0.0030 0.0360 0.0057
log_softmax 1768 55.1740 0.0150 0.0550 0.0312
_backward_copy 1768 91.5830 0.0220 0.0760 0.0518
_backward_Convolution 3536 32698.2793 2.6240 12.5610 9.2473
_mul_scalar 1768 6.3060 0.0020 0.0230 0.0036
_backward_log_softmax 1768 30.2240 0.0090 0.0400 0.0171
mean 5544 106.2690 0.0080 0.0730 0.0192
_backward_mean 1768 19.4110 0.0080 0.1030 0.0110
_backward_FullyConnected 5304 1018.3400 0.0280 0.5940 0.1920
WaitForVar 3776 33.5890 0.0060 0.0280 0.0089
broadcast_equal 2008 10.0300 0.0030 0.0110 0.0050
sgd_update 17680 177.3020 0.0020 0.0790 0.0100
_backward_Activation 7072 522.6880 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69090 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6030 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 884 3513.9780 0.3060 73.5870 3.9751
MXNDArrayCreateEx 1005 34.7970 0.0310 0.0640 0.0346
MXNDArrayWaitAll 885 19202.8340 5.6130 25.0610 21.6981
MXNDArraySyncCopyToCPU 1890 14806.6758 0.0600 24.1980 7.8342
MXNDArrayGetDType 1890 0.6870 0.0000 0.0010 0.0004
MXNDArrayGetShape 4785 2.2330 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2010 17.6820 0.0070 0.0280 0.0088
MXNDArrayFree 22515 231.2920 0.0000 0.9140 0.0103
MXNet C API Calls 101295 588.6520 487.3580 588.6520 50.6470
MXNet C API Concurrency 202589 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1770 0.8420 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8850 4.5700 0.0000 0.0230 0.0005
MXNDArraySetGradState 8850 4.5100 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1770 0.7550 0.0000 0.0010 0.0004
MXAutogradBackwardEx 885 489.5760 0.5250 0.9450 0.5532
MXNDArrayGetContext 13950 6.1390 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29355 755.5190 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3540 8652.5713 0.7010 3.7500 2.4442
FullyConnected 6030 582.6600 0.0080 0.3430 0.0966
argmax 2010 39.8630 0.0050 0.0480 0.0198
_backward_pick 1770 28.1140 0.0130 0.0320 0.0159
_backward_mul_scalar 1770 7.7370 0.0030 0.0220 0.0044
DeleteVariable 75120 533.3610 0.0010 0.9010 0.0071
Flatten 2010 83.8670 0.0080 0.1120 0.0417
Activation 8040 373.8480 0.0030 1.9440 0.0465
SetValueOp 1770 11.5060 0.0050 0.0180 0.0065
pick 1770 13.9260 0.0060 0.0280 0.0079
Pooling 4020 5584.8682 0.0640 3.1720 1.3893
Convolution 4020 22815.2656 0.2260 13.2050 5.6754
CopyCPU2CPU 2010 11.4620 0.0030 0.0360 0.0057
log_softmax 1770 55.2330 0.0150 0.0550 0.0312
_backward_copy 1770 91.6860 0.0220 0.0760 0.0518
_backward_Convolution 3540 32735.3711 2.6240 12.5610 9.2473
_mul_scalar 1770 6.3130 0.0020 0.0230 0.0036
_backward_log_softmax 1770 30.2550 0.0090 0.0400 0.0171
mean 5550 106.3840 0.0080 0.0730 0.0192
_backward_mean 1770 19.4320 0.0080 0.1030 0.0110
_backward_FullyConnected 5310 1019.4110 0.0280 0.5940 0.1920
WaitForVar 3780 33.6240 0.0060 0.0280 0.0089
broadcast_equal 2010 10.0400 0.0030 0.0110 0.0050
sgd_update 17700 177.5010 0.0020 0.0790 0.0100
_backward_Activation 7080 523.2330 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69164 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6036 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 885 3517.8799 0.3060 73.5870 3.9750
MXNDArrayCreateEx 1006 34.8310 0.0310 0.0640 0.0346
MXNDArrayWaitAll 886 19224.7051 5.6130 25.0610 21.6983
MXNDArraySyncCopyToCPU 1892 14821.1094 0.0600 24.1980 7.8336
MXNDArrayGetDType 1892 0.6890 0.0000 0.0010 0.0004
MXNDArrayGetShape 4790 2.2370 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2012 17.6990 0.0070 0.0280 0.0088
MXNDArrayFree 22538 231.5200 0.0000 0.9140 0.0103
MXNet C API Calls 101402 588.7590 487.3580 588.7590 50.7005
MXNet C API Concurrency 202803 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1772 0.8420 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8860 4.5750 0.0000 0.0230 0.0005
MXNDArraySetGradState 8860 4.5160 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1772 0.7570 0.0000 0.0010 0.0004
MXAutogradBackwardEx 886 490.1100 0.5250 0.9450 0.5532
MXNDArrayGetContext 13964 6.1440 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29386 756.2890 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3544 8662.3066 0.7010 3.7500 2.4442
FullyConnected 6036 583.1820 0.0080 0.3430 0.0966
argmax 2012 39.9000 0.0050 0.0480 0.0198
_backward_pick 1772 28.1450 0.0130 0.0320 0.0159
_backward_mul_scalar 1772 7.7460 0.0030 0.0220 0.0044
DeleteVariable 75200 533.9000 0.0010 0.9010 0.0071
Flatten 2012 83.9440 0.0080 0.1120 0.0417
Activation 8048 374.2000 0.0030 1.9440 0.0465
SetValueOp 1772 11.5180 0.0050 0.0180 0.0065
pick 1772 13.9410 0.0060 0.0280 0.0079
Pooling 4024 5590.2539 0.0640 3.1720 1.3892
Convolution 4024 22837.5098 0.2260 13.2050 5.6753
CopyCPU2CPU 2012 11.4710 0.0030 0.0360 0.0057
log_softmax 1772 55.2940 0.0150 0.0550 0.0312
_backward_copy 1772 91.7910 0.0220 0.0760 0.0518
_backward_Convolution 3544 32772.4453 2.6240 12.5610 9.2473
_mul_scalar 1772 6.3200 0.0020 0.0230 0.0036
_backward_log_softmax 1772 30.2870 0.0090 0.0400 0.0171
mean 5556 106.4950 0.0080 0.0730 0.0192
_backward_mean 1772 19.4530 0.0080 0.1030 0.0110
_backward_FullyConnected 5316 1020.4810 0.0280 0.5940 0.1920
WaitForVar 3784 33.6600 0.0060 0.0280 0.0089
broadcast_equal 2012 10.0500 0.0030 0.0110 0.0050
sgd_update 17720 177.7020 0.0020 0.0790 0.0100
_backward_Activation 7088 523.8110 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69238 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6042 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 886 3521.7690 0.3060 73.5870 3.9749
MXNDArrayCreateEx 1007 34.8640 0.0310 0.0640 0.0346
MXNDArrayWaitAll 887 19246.4102 5.6130 25.0610 21.6983
MXNDArraySyncCopyToCPU 1894 14835.6006 0.0600 24.1980 7.8329
MXNDArrayGetDType 1894 0.6900 0.0000 0.0010 0.0004
MXNDArrayGetShape 4795 2.2380 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2014 17.7160 0.0070 0.0280 0.0088
MXNDArrayFree 22561 231.7380 0.0000 0.9140 0.0103
MXNet C API Calls 101509 588.8660 487.3580 588.8660 50.7540
MXNet C API Concurrency 203017 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1774 0.8440 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8870 4.5800 0.0000 0.0230 0.0005
MXNDArraySetGradState 8870 4.5230 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1774 0.7580 0.0000 0.0010 0.0004
MXAutogradBackwardEx 887 490.6450 0.5250 0.9450 0.5532
MXNDArrayGetContext 13978 6.1480 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29417 757.0560 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3548 8672.0547 0.7010 3.7500 2.4442
FullyConnected 6042 583.7250 0.0080 0.3430 0.0966
argmax 2014 39.9380 0.0050 0.0480 0.0198
_backward_pick 1774 28.1760 0.0130 0.0320 0.0159
_backward_mul_scalar 1774 7.7550 0.0030 0.0220 0.0044
DeleteVariable 75280 534.4100 0.0010 0.9010 0.0071
Flatten 2014 84.0210 0.0080 0.1120 0.0417
Activation 8056 374.5510 0.0030 1.9440 0.0465
SetValueOp 1774 11.5310 0.0050 0.0180 0.0065
pick 1774 13.9570 0.0060 0.0280 0.0079
Pooling 4028 5595.6250 0.0640 3.1720 1.3892
Convolution 4028 22859.7129 0.2260 13.2050 5.6752
CopyCPU2CPU 2014 11.4800 0.0030 0.0360 0.0057
log_softmax 1774 55.3530 0.0150 0.0550 0.0312
_backward_copy 1774 91.9340 0.0220 0.0760 0.0518
_backward_Convolution 3548 32809.2695 2.6240 12.5610 9.2473
_mul_scalar 1774 6.3260 0.0020 0.0230 0.0036
_backward_log_softmax 1774 30.3200 0.0090 0.0400 0.0171
mean 5562 106.6060 0.0080 0.0730 0.0192
_backward_mean 1774 19.4750 0.0080 0.1030 0.0110
_backward_FullyConnected 5322 1021.5490 0.0280 0.5940 0.1919
WaitForVar 3788 33.6950 0.0060 0.0280 0.0089
broadcast_equal 2014 10.0590 0.0030 0.0110 0.0050
sgd_update 17740 177.9000 0.0020 0.0790 0.0100
_backward_Activation 7096 524.3580 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69312 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6048 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 887 3525.6670 0.3060 73.5870 3.9748
MXNDArrayCreateEx 1008 34.8970 0.0310 0.0640 0.0346
MXNDArrayWaitAll 888 19268.1641 5.6130 25.0610 21.6984
MXNDArraySyncCopyToCPU 1896 14850.3096 0.0600 24.1980 7.8324
MXNDArrayGetDType 1896 0.6910 0.0000 0.0010 0.0004
MXNDArrayGetShape 4800 2.2400 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2016 17.7320 0.0070 0.0280 0.0088
MXNDArrayFree 22584 231.9580 0.0000 0.9140 0.0103
MXNet C API Calls 101616 588.9730 487.3580 588.9730 50.8075
MXNet C API Concurrency 203231 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1776 0.8450 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8880 4.5830 0.0000 0.0230 0.0005
MXNDArraySetGradState 8880 4.5300 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1776 0.7590 0.0000 0.0010 0.0004
MXAutogradBackwardEx 888 491.1960 0.5250 0.9450 0.5531
MXNDArrayGetContext 13992 6.1520 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29448 757.8240 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3552 8681.6846 0.7010 3.7500 2.4442
FullyConnected 6048 584.4540 0.0080 0.3430 0.0966
argmax 2016 39.9840 0.0050 0.0480 0.0198
_backward_pick 1776 28.2090 0.0130 0.0320 0.0159
_backward_mul_scalar 1776 7.7640 0.0030 0.0220 0.0044
DeleteVariable 75360 534.9220 0.0010 0.9010 0.0071
Flatten 2016 84.0980 0.0080 0.1120 0.0417
Activation 8064 374.9250 0.0030 1.9440 0.0465
SetValueOp 1776 11.5430 0.0050 0.0180 0.0065
pick 1776 13.9730 0.0060 0.0280 0.0079
Pooling 4032 5600.9819 0.0640 3.1720 1.3891
Convolution 4032 22881.8965 0.2260 13.2050 5.6751
CopyCPU2CPU 2016 11.4890 0.0030 0.0360 0.0057
log_softmax 1776 55.4250 0.0150 0.0550 0.0312
_backward_copy 1776 92.0340 0.0220 0.0760 0.0518
_backward_Convolution 3552 32846.1523 2.6240 12.5610 9.2472
_mul_scalar 1776 6.3330 0.0020 0.0230 0.0036
_backward_log_softmax 1776 30.3630 0.0090 0.0400 0.0171
mean 5568 106.7110 0.0080 0.0730 0.0192
_backward_mean 1776 19.4980 0.0080 0.1030 0.0110
_backward_FullyConnected 5328 1022.9500 0.0280 0.5940 0.1920
WaitForVar 3792 33.7410 0.0060 0.0280 0.0089
broadcast_equal 2016 10.0700 0.0030 0.0110 0.0050
sgd_update 17760 178.1970 0.0020 0.0790 0.0100
_backward_Activation 7104 524.9400 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69386 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6054 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 888 3529.5681 0.3060 73.5870 3.9747
MXNDArrayCreateEx 1009 34.9300 0.0310 0.0640 0.0346
MXNDArrayWaitAll 889 19289.9590 5.6130 25.0610 21.6985
MXNDArraySyncCopyToCPU 1898 14864.7432 0.0600 24.1980 7.8318
MXNDArrayGetDType 1898 0.6920 0.0000 0.0010 0.0004
MXNDArrayGetShape 4805 2.2410 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2018 17.7490 0.0070 0.0280 0.0088
MXNDArrayFree 22607 232.1820 0.0000 0.9140 0.0103
MXNet C API Calls 101723 589.0800 487.3580 589.0800 50.8610
MXNet C API Concurrency 203445 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1778 0.8470 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8890 4.5870 0.0000 0.0230 0.0005
MXNDArraySetGradState 8890 4.5340 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1778 0.7590 0.0000 0.0010 0.0004
MXAutogradBackwardEx 889 491.7250 0.5250 0.9450 0.5531
MXNDArrayGetContext 14006 6.1570 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29479 758.5910 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3556 8691.3223 0.7010 3.7500 2.4441
FullyConnected 6054 584.9960 0.0080 0.3430 0.0966
argmax 2018 40.0210 0.0050 0.0480 0.0198
_backward_pick 1778 28.2400 0.0130 0.0320 0.0159
_backward_mul_scalar 1778 7.7730 0.0030 0.0220 0.0044
DeleteVariable 75440 535.4440 0.0010 0.9010 0.0071
Flatten 2018 84.1770 0.0080 0.1120 0.0417
Activation 8072 375.2770 0.0030 1.9440 0.0465
SetValueOp 1778 11.5560 0.0050 0.0180 0.0065
pick 1778 13.9880 0.0060 0.0280 0.0079
Pooling 4036 5606.3838 0.0640 3.1720 1.3891
Convolution 4036 22904.0859 0.2260 13.2050 5.6749
CopyCPU2CPU 2018 11.4990 0.0030 0.0360 0.0057
log_softmax 1778 55.4840 0.0150 0.0550 0.0312
_backward_copy 1778 92.1390 0.0220 0.0760 0.0518
_backward_Convolution 3556 32883.2031 2.6240 12.5610 9.2472
_mul_scalar 1778 6.3400 0.0020 0.0230 0.0036
_backward_log_softmax 1778 30.3950 0.0090 0.0400 0.0171
mean 5574 106.8230 0.0080 0.0730 0.0192
_backward_mean 1778 19.5190 0.0080 0.1030 0.0110
_backward_FullyConnected 5334 1024.0200 0.0280 0.5940 0.1920
WaitForVar 3796 33.7760 0.0060 0.0280 0.0089
broadcast_equal 2018 10.0790 0.0030 0.0110 0.0050
sgd_update 17780 178.3910 0.0020 0.0790 0.0100
_backward_Activation 7112 525.4880 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69460 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6060 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 889 3533.4661 0.3060 73.5870 3.9747
MXNDArrayCreateEx 1010 34.9630 0.0310 0.0640 0.0346
MXNDArrayWaitAll 890 19311.8594 5.6130 25.0610 21.6987
MXNDArraySyncCopyToCPU 1900 14879.1143 0.0600 24.1980 7.8311
MXNDArrayGetDType 1900 0.6930 0.0000 0.0010 0.0004
MXNDArrayGetShape 4810 2.2430 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2020 17.7670 0.0070 0.0280 0.0088
MXNDArrayFree 22630 232.4070 0.0000 0.9140 0.0103
MXNet C API Calls 101830 589.1870 487.3580 589.1870 50.9145
MXNet C API Concurrency 203659 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1780 0.8470 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8900 4.5940 0.0000 0.0230 0.0005
MXNDArraySetGradState 8900 4.5400 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1780 0.7600 0.0000 0.0010 0.0004
MXAutogradBackwardEx 890 492.2540 0.5250 0.9450 0.5531
MXNDArrayGetContext 14020 6.1620 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29510 759.3620 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3560 8701.0098 0.7010 3.7500 2.4441
FullyConnected 6060 585.5260 0.0080 0.3430 0.0966
argmax 2020 40.0580 0.0050 0.0480 0.0198
_backward_pick 1780 28.2700 0.0130 0.0320 0.0159
_backward_mul_scalar 1780 7.7830 0.0030 0.0220 0.0044
DeleteVariable 75520 535.9520 0.0010 0.9010 0.0071
Flatten 2020 84.2540 0.0080 0.1120 0.0417
Activation 8080 375.6290 0.0030 1.9440 0.0465
SetValueOp 1780 11.5690 0.0050 0.0180 0.0065
pick 1780 14.0040 0.0060 0.0280 0.0079
Pooling 4040 5611.7310 0.0640 3.1720 1.3890
Convolution 4040 22926.3242 0.2260 13.2050 5.6748
CopyCPU2CPU 2020 11.5080 0.0030 0.0360 0.0057
log_softmax 1780 55.5430 0.0150 0.0550 0.0312
_backward_copy 1780 92.2420 0.0220 0.0760 0.0518
_backward_Convolution 3560 32920.3125 2.6240 12.5610 9.2473
_mul_scalar 1780 6.3470 0.0020 0.0230 0.0036
_backward_log_softmax 1780 30.4270 0.0090 0.0400 0.0171
mean 5580 106.9360 0.0080 0.0730 0.0192
_backward_mean 1780 19.5400 0.0080 0.1030 0.0110
_backward_FullyConnected 5340 1025.0861 0.0280 0.5940 0.1920
WaitForVar 3800 33.8130 0.0060 0.0280 0.0089
broadcast_equal 2020 10.0880 0.0030 0.0110 0.0050
sgd_update 17800 178.5390 0.0020 0.0790 0.0100
_backward_Activation 7120 526.0970 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69534 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6066 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 890 3537.3579 0.3060 73.5870 3.9746
MXNDArrayCreateEx 1011 34.9970 0.0310 0.0640 0.0346
MXNDArrayWaitAll 891 19333.5547 5.6130 25.0610 21.6987
MXNDArraySyncCopyToCPU 1902 14893.5312 0.0600 24.1980 7.8305
MXNDArrayGetDType 1902 0.6940 0.0000 0.0010 0.0004
MXNDArrayGetShape 4815 2.2450 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2022 17.7840 0.0070 0.0280 0.0088
MXNDArrayFree 22653 232.6360 0.0000 0.9140 0.0103
MXNet C API Calls 101937 589.2940 487.3580 589.2940 50.9680
MXNet C API Concurrency 203873 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1782 0.8480 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8910 4.5980 0.0000 0.0230 0.0005
MXNDArraySetGradState 8910 4.5440 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1782 0.7620 0.0000 0.0010 0.0004
MXAutogradBackwardEx 891 492.7860 0.5250 0.9450 0.5531
MXNDArrayGetContext 14034 6.1720 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29541 760.1380 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3564 8710.7168 0.7010 3.7500 2.4441
FullyConnected 6066 586.0690 0.0080 0.3430 0.0966
argmax 2022 40.0960 0.0050 0.0480 0.0198
_backward_pick 1782 28.3020 0.0130 0.0320 0.0159
_backward_mul_scalar 1782 7.7910 0.0030 0.0220 0.0044
DeleteVariable 75600 536.4770 0.0010 0.9010 0.0071
Flatten 2022 84.3350 0.0080 0.1120 0.0417
Activation 8088 375.9800 0.0030 1.9440 0.0465
SetValueOp 1782 11.5820 0.0050 0.0180 0.0065
pick 1782 14.0190 0.0060 0.0280 0.0079
Pooling 4044 5617.1138 0.0640 3.1720 1.3890
Convolution 4044 22948.5254 0.2260 13.2050 5.6747
CopyCPU2CPU 2022 11.5170 0.0030 0.0360 0.0057
log_softmax 1782 55.6030 0.0150 0.0550 0.0312
_backward_copy 1782 92.3450 0.0220 0.0760 0.0518
_backward_Convolution 3564 32957.0859 2.6240 12.5610 9.2472
_mul_scalar 1782 6.3530 0.0020 0.0230 0.0036
_backward_log_softmax 1782 30.4590 0.0090 0.0400 0.0171
mean 5586 107.0490 0.0080 0.0730 0.0192
_backward_mean 1782 19.5620 0.0080 0.1030 0.0110
_backward_FullyConnected 5346 1026.1510 0.0280 0.5940 0.1919
WaitForVar 3804 33.8480 0.0060 0.0280 0.0089
broadcast_equal 2022 10.0970 0.0030 0.0110 0.0050
sgd_update 17820 178.7410 0.0020 0.0790 0.0100
_backward_Activation 7128 526.6590 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69608 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6072 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 891 3541.2671 0.3060 73.5870 3.9745
MXNDArrayCreateEx 1012 35.0300 0.0310 0.0640 0.0346
MXNDArrayWaitAll 892 19355.2090 5.6130 25.0610 21.6987
MXNDArraySyncCopyToCPU 1904 14908.3506 0.0600 24.1980 7.8300
MXNDArrayGetDType 1904 0.6940 0.0000 0.0010 0.0004
MXNDArrayGetShape 4820 2.2480 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2024 17.8020 0.0070 0.0280 0.0088
MXNDArrayFree 22676 232.8640 0.0000 0.9140 0.0103
MXNet C API Calls 102044 589.4010 487.3580 589.4010 51.0215
MXNet C API Concurrency 204087 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1784 0.8490 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8920 4.6020 0.0000 0.0230 0.0005
MXNDArraySetGradState 8920 4.5510 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1784 0.7630 0.0000 0.0010 0.0004
MXAutogradBackwardEx 892 493.3150 0.5250 0.9450 0.5530
MXNDArrayGetContext 14048 6.1810 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29572 760.9080 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3568 8720.4082 0.7010 3.7500 2.4441
FullyConnected 6072 586.7930 0.0080 0.3430 0.0966
argmax 2024 40.1420 0.0050 0.0480 0.0198
_backward_pick 1784 28.3350 0.0130 0.0320 0.0159
_backward_mul_scalar 1784 7.7990 0.0030 0.0220 0.0044
DeleteVariable 75680 537.0030 0.0010 0.9010 0.0071
Flatten 2024 84.4140 0.0080 0.1120 0.0417
Activation 8096 376.3640 0.0030 1.9440 0.0465
SetValueOp 1784 11.5950 0.0050 0.0180 0.0065
pick 1784 14.0360 0.0060 0.0280 0.0079
Pooling 4048 5622.4668 0.0640 3.1720 1.3889
Convolution 4048 22970.7227 0.2260 13.2050 5.6746
CopyCPU2CPU 2024 11.5260 0.0030 0.0360 0.0057
log_softmax 1784 55.6760 0.0150 0.0550 0.0312
_backward_copy 1784 92.4440 0.0220 0.0760 0.0518
_backward_Convolution 3568 32993.8711 2.6240 12.5610 9.2472
_mul_scalar 1784 6.3610 0.0020 0.0230 0.0036
_backward_log_softmax 1784 30.5010 0.0090 0.0400 0.0171
mean 5592 107.1560 0.0080 0.0730 0.0192
_backward_mean 1784 19.5830 0.0080 0.1030 0.0110
_backward_FullyConnected 5352 1027.6060 0.0280 0.5940 0.1920
WaitForVar 3808 33.8830 0.0060 0.0280 0.0089
broadcast_equal 2024 10.1070 0.0030 0.0110 0.0050
sgd_update 17840 179.0170 0.0020 0.0790 0.0100
_backward_Activation 7136 527.2420 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69682 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6078 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 892 3545.1660 0.3060 73.5870 3.9744
MXNDArrayCreateEx 1013 35.0640 0.0310 0.0640 0.0346
MXNDArrayWaitAll 893 19377.0352 5.6130 25.0610 21.6988
MXNDArraySyncCopyToCPU 1906 14922.6855 0.0600 24.1980 7.8293
MXNDArrayGetDType 1906 0.6940 0.0000 0.0010 0.0004
MXNDArrayGetShape 4825 2.2500 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2026 17.8180 0.0070 0.0280 0.0088
MXNDArrayFree 22699 233.0940 0.0000 0.9140 0.0103
MXNet C API Calls 102151 589.5080 487.3580 589.5080 51.0750
MXNet C API Concurrency 204301 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1786 0.8490 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8930 4.6080 0.0000 0.0230 0.0005
MXNDArraySetGradState 8930 4.5550 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1786 0.7630 0.0000 0.0010 0.0004
MXAutogradBackwardEx 893 493.8690 0.5250 0.9450 0.5530
MXNDArrayGetContext 14062 6.1870 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29603 761.6730 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3572 8729.9629 0.7010 3.7500 2.4440
FullyConnected 6078 587.3260 0.0080 0.3430 0.0966
argmax 2026 40.1800 0.0050 0.0480 0.0198
_backward_pick 1786 28.3650 0.0130 0.0320 0.0159
_backward_mul_scalar 1786 7.8080 0.0030 0.0220 0.0044
DeleteVariable 75760 537.5260 0.0010 0.9010 0.0071
Flatten 2026 84.4920 0.0080 0.1120 0.0417
Activation 8104 376.7140 0.0030 1.9440 0.0465
SetValueOp 1786 11.6080 0.0050 0.0180 0.0065
pick 1786 14.0520 0.0060 0.0280 0.0079
Pooling 4052 5627.8970 0.0640 3.1720 1.3889
Convolution 4052 22992.8477 0.2260 13.2050 5.6744
CopyCPU2CPU 2026 11.5370 0.0030 0.0360 0.0057
log_softmax 1786 55.7350 0.0150 0.0550 0.0312
_backward_copy 1786 92.5490 0.0220 0.0760 0.0518
_backward_Convolution 3572 33030.9531 2.6240 12.5610 9.2472
_mul_scalar 1786 6.3690 0.0020 0.0230 0.0036
_backward_log_softmax 1786 30.5330 0.0090 0.0400 0.0171
mean 5598 107.2680 0.0080 0.0730 0.0192
_backward_mean 1786 19.6050 0.0080 0.1030 0.0110
_backward_FullyConnected 5358 1028.6730 0.0280 0.5940 0.1920
WaitForVar 3812 33.9190 0.0060 0.0280 0.0089
broadcast_equal 2026 10.1160 0.0030 0.0110 0.0050
sgd_update 17860 179.1680 0.0020 0.0790 0.0100
_backward_Activation 7144 527.8340 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69756 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6084 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 893 3549.0811 0.3060 73.5870 3.9743
MXNDArrayCreateEx 1014 35.0970 0.0310 0.0640 0.0346
MXNDArrayWaitAll 894 19398.9043 5.6130 25.0610 21.6990
MXNDArraySyncCopyToCPU 1908 14937.1367 0.0600 24.1980 7.8287
MXNDArrayGetDType 1908 0.6940 0.0000 0.0010 0.0004
MXNDArrayGetShape 4830 2.2510 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2028 17.8350 0.0070 0.0280 0.0088
MXNDArrayFree 22722 233.3270 0.0000 0.9140 0.0103
MXNet C API Calls 102258 589.6150 487.3580 589.6150 51.1285
MXNet C API Concurrency 204515 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1788 0.8510 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8940 4.6120 0.0000 0.0230 0.0005
MXNDArraySetGradState 8940 4.5610 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1788 0.7650 0.0000 0.0010 0.0004
MXAutogradBackwardEx 894 494.4020 0.5250 0.9450 0.5530
MXNDArrayGetContext 14076 6.1920 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29634 762.4460 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3576 8739.7090 0.7010 3.7500 2.4440
FullyConnected 6084 587.8700 0.0080 0.3430 0.0966
argmax 2028 40.2190 0.0050 0.0480 0.0198
_backward_pick 1788 28.3940 0.0130 0.0320 0.0159
_backward_mul_scalar 1788 7.8150 0.0030 0.0220 0.0044
DeleteVariable 75840 538.0490 0.0010 0.9010 0.0071
Flatten 2028 84.5700 0.0080 0.1120 0.0417
Activation 8112 377.0670 0.0030 1.9440 0.0465
SetValueOp 1788 11.6220 0.0050 0.0180 0.0065
pick 1788 14.0680 0.0060 0.0280 0.0079
Pooling 4056 5633.2759 0.0640 3.1720 1.3889
Convolution 4056 23015.0840 0.2260 13.2050 5.6743
CopyCPU2CPU 2028 11.5470 0.0030 0.0360 0.0057
log_softmax 1788 55.7940 0.0150 0.0550 0.0312
_backward_copy 1788 92.6520 0.0220 0.0760 0.0518
_backward_Convolution 3576 33068.0039 2.6240 12.5610 9.2472
_mul_scalar 1788 6.3760 0.0020 0.0230 0.0036
_backward_log_softmax 1788 30.5650 0.0090 0.0400 0.0171
mean 5604 107.3790 0.0080 0.0730 0.0192
_backward_mean 1788 19.6260 0.0080 0.1030 0.0110
_backward_FullyConnected 5364 1029.7390 0.0280 0.5940 0.1920
WaitForVar 3816 33.9570 0.0060 0.0280 0.0089
broadcast_equal 2028 10.1250 0.0030 0.0110 0.0050
sgd_update 17880 179.3730 0.0020 0.0790 0.0100
_backward_Activation 7152 528.3970 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69830 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6090 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 894 3552.9810 0.3060 73.5870 3.9743
MXNDArrayCreateEx 1015 35.1300 0.0310 0.0640 0.0346
MXNDArrayWaitAll 895 19420.5938 5.6130 25.0610 21.6990
MXNDArraySyncCopyToCPU 1910 14951.3193 0.0600 24.1980 7.8279
MXNDArrayGetDType 1910 0.6940 0.0000 0.0010 0.0004
MXNDArrayGetShape 4835 2.2520 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2030 17.8520 0.0070 0.0280 0.0088
MXNDArrayFree 22745 233.5510 0.0000 0.9140 0.0103
MXNet C API Calls 102365 589.7220 487.3580 589.7220 51.1820
MXNet C API Concurrency 204729 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1790 0.8510 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8950 4.6180 0.0000 0.0230 0.0005
MXNDArraySetGradState 8950 4.5640 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1790 0.7660 0.0000 0.0010 0.0004
MXAutogradBackwardEx 895 494.9550 0.5250 0.9450 0.5530
MXNDArrayGetContext 14090 6.2020 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29665 763.2440 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3580 8749.2344 0.7010 3.7500 2.4439
FullyConnected 6090 588.4330 0.0080 0.3430 0.0966
argmax 2030 40.2570 0.0050 0.0480 0.0198
_backward_pick 1790 28.4240 0.0130 0.0320 0.0159
_backward_mul_scalar 1790 7.8230 0.0030 0.0220 0.0044
DeleteVariable 75920 538.5750 0.0010 0.9010 0.0071
Flatten 2030 84.6470 0.0080 0.1120 0.0417
Activation 8120 377.4200 0.0030 1.9440 0.0465
SetValueOp 1790 11.6350 0.0050 0.0180 0.0065
pick 1790 14.0830 0.0060 0.0280 0.0079
Pooling 4060 5638.6118 0.0640 3.1720 1.3888
Convolution 4060 23037.2637 0.2260 13.2050 5.6742
CopyCPU2CPU 2030 11.5570 0.0030 0.0360 0.0057
log_softmax 1790 55.8540 0.0150 0.0550 0.0312
_backward_copy 1790 92.7550 0.0220 0.0760 0.0518
_backward_Convolution 3580 33104.8398 2.6240 12.5610 9.2472
_mul_scalar 1790 6.3830 0.0020 0.0230 0.0036
_backward_log_softmax 1790 30.5960 0.0090 0.0400 0.0171
mean 5610 107.4700 0.0080 0.0730 0.0192
_backward_mean 1790 19.6470 0.0080 0.1030 0.0110
_backward_FullyConnected 5370 1030.8051 0.0280 0.5940 0.1920
WaitForVar 3820 33.9950 0.0060 0.0280 0.0089
broadcast_equal 2030 10.1340 0.0030 0.0110 0.0050
sgd_update 17900 179.5740 0.0020 0.0790 0.0100
_backward_Activation 7160 528.9560 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69904 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6096 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 895 3556.8870 0.3060 73.5870 3.9742
MXNDArrayCreateEx 1016 35.1640 0.0310 0.0640 0.0346
MXNDArrayWaitAll 896 19442.2754 5.6130 25.0610 21.6990
MXNDArraySyncCopyToCPU 1912 14966.0195 0.0600 24.1980 7.8274
MXNDArrayGetDType 1912 0.6940 0.0000 0.0010 0.0004
MXNDArrayGetShape 4840 2.2540 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2032 17.8680 0.0070 0.0280 0.0088
MXNDArrayFree 22768 233.7720 0.0000 0.9140 0.0103
MXNet C API Calls 102472 589.8290 487.3580 589.8290 51.2355
MXNet C API Concurrency 204943 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1792 0.8510 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8960 4.6220 0.0000 0.0230 0.0005
MXNDArraySetGradState 8960 4.5700 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1792 0.7670 0.0000 0.0010 0.0004
MXAutogradBackwardEx 896 495.4820 0.5250 0.9450 0.5530
MXNDArrayGetContext 14104 6.2050 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29696 764.0180 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3584 8758.7998 0.7010 3.7500 2.4439
FullyConnected 6096 589.1580 0.0080 0.3430 0.0966
argmax 2032 40.3010 0.0050 0.0480 0.0198
_backward_pick 1792 28.4570 0.0130 0.0320 0.0159
_backward_mul_scalar 1792 7.8320 0.0030 0.0220 0.0044
DeleteVariable 76000 539.0960 0.0010 0.9010 0.0071
Flatten 2032 84.7260 0.0080 0.1120 0.0417
Activation 8128 377.8030 0.0030 1.9440 0.0465
SetValueOp 1792 11.6490 0.0050 0.0180 0.0065
pick 1792 14.0990 0.0060 0.0280 0.0079
Pooling 4064 5643.9668 0.0640 3.1720 1.3888
Convolution 4064 23059.4375 0.2260 13.2050 5.6741
CopyCPU2CPU 2032 11.5670 0.0030 0.0360 0.0057
log_softmax 1792 55.9260 0.0150 0.0550 0.0312
_backward_copy 1792 92.8550 0.0220 0.0760 0.0518
_backward_Convolution 3584 33141.6328 2.6240 12.5610 9.2471
_mul_scalar 1792 6.3900 0.0020 0.0230 0.0036
_backward_log_softmax 1792 30.6380 0.0090 0.0400 0.0171
mean 5616 107.5760 0.0080 0.0730 0.0192
_backward_mean 1792 19.6680 0.0080 0.1030 0.0110
_backward_FullyConnected 5376 1032.2111 0.0280 0.5940 0.1920
WaitForVar 3824 34.0300 0.0060 0.0280 0.0089
broadcast_equal 2032 10.1430 0.0030 0.0110 0.0050
sgd_update 17920 179.7880 0.0020 0.0790 0.0100
_backward_Activation 7168 529.6190 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 69978 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6102 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 896 3560.7949 0.3060 73.5870 3.9741
MXNDArrayCreateEx 1017 35.1970 0.0310 0.0640 0.0346
MXNDArrayWaitAll 897 19464.1230 5.6130 25.0610 21.6991
MXNDArraySyncCopyToCPU 1914 14980.4072 0.0600 24.1980 7.8268
MXNDArrayGetDType 1914 0.6950 0.0000 0.0010 0.0004
MXNDArrayGetShape 4845 2.2550 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2034 17.8850 0.0070 0.0280 0.0088
MXNDArrayFree 22791 233.9980 0.0000 0.9140 0.0103
MXNet C API Calls 102579 589.9360 487.3580 589.9360 51.2890
MXNet C API Concurrency 205157 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1794 0.8510 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8970 4.6280 0.0000 0.0230 0.0005
MXNDArraySetGradState 8970 4.5750 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1794 0.7680 0.0000 0.0010 0.0004
MXAutogradBackwardEx 897 496.0110 0.5250 0.9450 0.5530
MXNDArrayGetContext 14118 6.2120 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29727 764.8120 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3588 8768.4551 0.7010 3.7500 2.4438
FullyConnected 6102 589.7020 0.0080 0.3430 0.0966
argmax 2034 40.3400 0.0050 0.0480 0.0198
_backward_pick 1794 28.4880 0.0130 0.0320 0.0159
_backward_mul_scalar 1794 7.8410 0.0030 0.0220 0.0044
DeleteVariable 76080 539.6090 0.0010 0.9010 0.0071
Flatten 2034 84.8060 0.0080 0.1120 0.0417
Activation 8136 378.1550 0.0030 1.9440 0.0465
SetValueOp 1794 11.6620 0.0050 0.0180 0.0065
pick 1794 14.1130 0.0060 0.0280 0.0079
Pooling 4068 5649.2900 0.0640 3.1720 1.3887
Convolution 4068 23081.6035 0.2260 13.2050 5.6739
CopyCPU2CPU 2034 11.5770 0.0030 0.0360 0.0057
log_softmax 1794 55.9860 0.0150 0.0550 0.0312
_backward_copy 1794 92.9580 0.0220 0.0760 0.0518
_backward_Convolution 3588 33178.7188 2.6240 12.5610 9.2471
_mul_scalar 1794 6.3970 0.0020 0.0230 0.0036
_backward_log_softmax 1794 30.6700 0.0090 0.0400 0.0171
mean 5622 107.6950 0.0080 0.0730 0.0192
_backward_mean 1794 19.6890 0.0080 0.1030 0.0110
_backward_FullyConnected 5382 1033.3270 0.0280 0.5940 0.1920
WaitForVar 3828 34.0650 0.0060 0.0280 0.0089
broadcast_equal 2034 10.1540 0.0030 0.0110 0.0050
sgd_update 17940 179.9930 0.0020 0.0790 0.0100
_backward_Activation 7176 530.1780 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70052 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6108 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 897 3564.6919 0.3060 73.5870 3.9740
MXNDArrayCreateEx 1018 35.2300 0.0310 0.0640 0.0346
MXNDArrayWaitAll 898 19485.8594 5.6130 25.0610 21.6992
MXNDArraySyncCopyToCPU 1916 14994.8105 0.0600 24.1980 7.8261
MXNDArrayGetDType 1916 0.6960 0.0000 0.0010 0.0004
MXNDArrayGetShape 4850 2.2570 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2036 17.9010 0.0070 0.0280 0.0088
MXNDArrayFree 22814 234.2290 0.0000 0.9140 0.0103
MXNet C API Calls 102686 590.0430 487.3580 590.0430 51.3425
MXNet C API Concurrency 205371 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1796 0.8510 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8980 4.6320 0.0000 0.0230 0.0005
MXNDArraySetGradState 8980 4.5800 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1796 0.7690 0.0000 0.0010 0.0004
MXAutogradBackwardEx 898 496.5710 0.5250 0.9450 0.5530
MXNDArrayGetContext 14132 6.2200 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29758 765.5780 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3592 8778.1289 0.7010 3.7500 2.4438
FullyConnected 6108 590.2510 0.0080 0.3430 0.0966
argmax 2036 40.3770 0.0050 0.0480 0.0198
_backward_pick 1796 28.5180 0.0130 0.0320 0.0159
_backward_mul_scalar 1796 7.8490 0.0030 0.0220 0.0044
DeleteVariable 76160 540.1320 0.0010 0.9010 0.0071
Flatten 2036 84.8850 0.0080 0.1120 0.0417
Activation 8144 378.5090 0.0030 1.9440 0.0465
SetValueOp 1796 11.6750 0.0050 0.0180 0.0065
pick 1796 14.1280 0.0060 0.0280 0.0079
Pooling 4072 5654.6860 0.0640 3.1720 1.3887
Convolution 4072 23103.8125 0.2260 13.2050 5.6738
CopyCPU2CPU 2036 11.5870 0.0030 0.0360 0.0057
log_softmax 1796 56.0440 0.0150 0.0550 0.0312
_backward_copy 1796 93.0620 0.0220 0.0760 0.0518
_backward_Convolution 3592 33215.8555 2.6240 12.5610 9.2472
_mul_scalar 1796 6.4040 0.0020 0.0230 0.0036
_backward_log_softmax 1796 30.7020 0.0090 0.0400 0.0171
mean 5628 107.8160 0.0080 0.0730 0.0192
_backward_mean 1796 19.7100 0.0080 0.1030 0.0110
_backward_FullyConnected 5388 1034.3950 0.0280 0.5940 0.1920
WaitForVar 3832 34.1020 0.0060 0.0280 0.0089
broadcast_equal 2036 10.1640 0.0030 0.0110 0.0050
sgd_update 17960 180.1430 0.0020 0.0790 0.0100
_backward_Activation 7184 530.7680 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70126 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6114 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 898 3568.5930 0.3060 73.5870 3.9739
MXNDArrayCreateEx 1019 35.2630 0.0310 0.0640 0.0346
MXNDArrayWaitAll 899 19507.5957 5.6130 25.0610 21.6992
MXNDArraySyncCopyToCPU 1918 15009.2275 0.0600 24.1980 7.8255
MXNDArrayGetDType 1918 0.6960 0.0000 0.0010 0.0004
MXNDArrayGetShape 4855 2.2580 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2038 17.9180 0.0070 0.0280 0.0088
MXNDArrayFree 22837 234.4530 0.0000 0.9140 0.0103
MXNet C API Calls 102793 590.1500 487.3580 590.1500 51.3960
MXNet C API Concurrency 205585 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1798 0.8520 0.0000 0.0040 0.0005
MXNDArrayGetGradState 8990 4.6380 0.0000 0.0230 0.0005
MXNDArraySetGradState 8990 4.5870 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1798 0.7700 0.0000 0.0010 0.0004
MXAutogradBackwardEx 899 497.1010 0.5250 0.9450 0.5529
MXNDArrayGetContext 14146 6.2290 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29789 766.3450 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3596 8787.8467 0.7010 3.7500 2.4438
FullyConnected 6114 590.7980 0.0080 0.3430 0.0966
argmax 2038 40.4150 0.0050 0.0480 0.0198
_backward_pick 1798 28.5470 0.0130 0.0320 0.0159
_backward_mul_scalar 1798 7.8580 0.0030 0.0220 0.0044
DeleteVariable 76240 540.6410 0.0010 0.9010 0.0071
Flatten 2038 84.9650 0.0080 0.1120 0.0417
Activation 8152 378.8620 0.0030 1.9440 0.0465
SetValueOp 1798 11.6880 0.0050 0.0180 0.0065
pick 1798 14.1440 0.0060 0.0280 0.0079
Pooling 4076 5660.0518 0.0640 3.1720 1.3886
Convolution 4076 23125.9746 0.2260 13.2050 5.6737
CopyCPU2CPU 2038 11.5960 0.0030 0.0360 0.0057
log_softmax 1798 56.1040 0.0150 0.0550 0.0312
_backward_copy 1798 93.1660 0.0220 0.0760 0.0518
_backward_Convolution 3596 33252.6523 2.6240 12.5610 9.2471
_mul_scalar 1798 6.4110 0.0020 0.0230 0.0036
_backward_log_softmax 1798 30.7340 0.0090 0.0400 0.0171
mean 5634 107.9350 0.0080 0.0730 0.0192
_backward_mean 1798 19.7320 0.0080 0.1030 0.0110
_backward_FullyConnected 5394 1035.4600 0.0280 0.5940 0.1920
WaitForVar 3836 34.1370 0.0060 0.0280 0.0089
broadcast_equal 2038 10.1730 0.0030 0.0110 0.0050
sgd_update 17980 180.2900 0.0020 0.0790 0.0100
_backward_Activation 7192 531.3800 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70200 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6120 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 899 3572.5330 0.3060 73.5870 3.9739
MXNDArrayCreateEx 1020 35.2970 0.0310 0.0640 0.0346
MXNDArrayWaitAll 900 19529.3281 5.6130 25.0610 21.6993
MXNDArraySyncCopyToCPU 1920 15023.9199 0.0600 24.1980 7.8250
MXNDArrayGetDType 1920 0.6960 0.0000 0.0010 0.0004
MXNDArrayGetShape 4860 2.2600 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2040 17.9340 0.0070 0.0280 0.0088
MXNDArrayFree 22860 234.6780 0.0000 0.9140 0.0103
MXNet C API Calls 102900 590.2570 487.3580 590.2570 51.4495
MXNet C API Concurrency 205799 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1800 0.8520 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9000 4.6410 0.0000 0.0230 0.0005
MXNDArraySetGradState 9000 4.5940 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1800 0.7720 0.0000 0.0010 0.0004
MXAutogradBackwardEx 900 497.6340 0.5250 0.9450 0.5529
MXNDArrayGetContext 14160 6.2380 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29820 767.1190 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3600 8797.5361 0.7010 3.7500 2.4438
FullyConnected 6120 591.5680 0.0080 0.3430 0.0967
argmax 2040 40.4590 0.0050 0.0480 0.0198
_backward_pick 1800 28.5800 0.0130 0.0320 0.0159
_backward_mul_scalar 1800 7.8670 0.0030 0.0220 0.0044
DeleteVariable 76320 541.1840 0.0010 0.9010 0.0071
Flatten 2040 85.0450 0.0080 0.1120 0.0417
Activation 8160 379.2420 0.0030 1.9440 0.0465
SetValueOp 1800 11.7010 0.0050 0.0180 0.0065
pick 1800 14.1610 0.0060 0.0280 0.0079
Pooling 4080 5665.4082 0.0640 3.1720 1.3886
Convolution 4080 23148.1758 0.2260 13.2050 5.6736
CopyCPU2CPU 2040 11.6050 0.0030 0.0360 0.0057
log_softmax 1800 56.1760 0.0150 0.0550 0.0312
_backward_copy 1800 93.2670 0.0220 0.0760 0.0518
_backward_Convolution 3600 33289.3906 2.6240 12.5610 9.2471
_mul_scalar 1800 6.4180 0.0020 0.0230 0.0036
_backward_log_softmax 1800 30.7750 0.0090 0.0400 0.0171
mean 5640 108.0450 0.0080 0.0730 0.0192
_backward_mean 1800 19.7530 0.0080 0.1030 0.0110
_backward_FullyConnected 5400 1036.8600 0.0280 0.5940 0.1920
WaitForVar 3840 34.1720 0.0060 0.0280 0.0089
broadcast_equal 2040 10.1820 0.0030 0.0110 0.0050
sgd_update 18000 180.5590 0.0020 0.0790 0.0100
_backward_Activation 7200 531.9870 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70274 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6126 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 900 3576.4180 0.3060 73.5870 3.9738
MXNDArrayCreateEx 1021 35.3290 0.0310 0.0640 0.0346
MXNDArrayWaitAll 901 19551.1523 5.6130 25.0610 21.6994
MXNDArraySyncCopyToCPU 1922 15038.3564 0.0600 24.1980 7.8243
MXNDArrayGetDType 1922 0.6980 0.0000 0.0010 0.0004
MXNDArrayGetShape 4865 2.2640 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2042 17.9510 0.0070 0.0280 0.0088
MXNDArrayFree 22883 234.9010 0.0000 0.9140 0.0103
MXNet C API Calls 103007 590.3640 487.3580 590.3640 51.5030
MXNet C API Concurrency 206013 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1802 0.8520 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9010 4.6470 0.0000 0.0230 0.0005
MXNDArraySetGradState 9010 4.5990 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1802 0.7730 0.0000 0.0010 0.0004
MXAutogradBackwardEx 901 498.1670 0.5250 0.9450 0.5529
MXNDArrayGetContext 14174 6.2460 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29851 767.8850 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3604 8807.2490 0.7010 3.7500 2.4437
FullyConnected 6126 592.1130 0.0080 0.3430 0.0967
argmax 2042 40.4970 0.0050 0.0480 0.0198
_backward_pick 1802 28.6120 0.0130 0.0320 0.0159
_backward_mul_scalar 1802 7.8760 0.0030 0.0220 0.0044
DeleteVariable 76400 541.7100 0.0010 0.9010 0.0071
Flatten 2042 85.1230 0.0080 0.1120 0.0417
Activation 8168 379.5910 0.0030 1.9440 0.0465
SetValueOp 1802 11.7140 0.0050 0.0180 0.0065
pick 1802 14.1810 0.0060 0.0280 0.0079
Pooling 4084 5670.7900 0.0640 3.1720 1.3885
Convolution 4084 23170.3418 0.2260 13.2050 5.6734
CopyCPU2CPU 2042 11.6150 0.0030 0.0360 0.0057
log_softmax 1802 56.2350 0.0150 0.0550 0.0312
_backward_copy 1802 93.3700 0.0220 0.0760 0.0518
_backward_Convolution 3604 33326.3711 2.6240 12.5610 9.2471
_mul_scalar 1802 6.4250 0.0020 0.0230 0.0036
_backward_log_softmax 1802 30.8080 0.0090 0.0400 0.0171
mean 5646 108.1610 0.0080 0.0730 0.0192
_backward_mean 1802 19.7740 0.0080 0.1030 0.0110
_backward_FullyConnected 5406 1037.9260 0.0280 0.5940 0.1920
WaitForVar 3844 34.2090 0.0060 0.0280 0.0089
broadcast_equal 2042 10.1920 0.0030 0.0110 0.0050
sgd_update 18020 180.7070 0.0020 0.0790 0.0100
_backward_Activation 7208 532.6130 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70348 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6132 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 901 3580.3210 0.3060 73.5870 3.9737
MXNDArrayCreateEx 1022 35.3620 0.0310 0.0640 0.0346
MXNDArrayWaitAll 902 19573.0020 5.6130 25.0610 21.6996
MXNDArraySyncCopyToCPU 1924 15052.7393 0.0600 24.1980 7.8237
MXNDArrayGetDType 1924 0.6980 0.0000 0.0010 0.0004
MXNDArrayGetShape 4870 2.2660 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2044 17.9680 0.0070 0.0280 0.0088
MXNDArrayFree 22906 235.1330 0.0000 0.9140 0.0103
MXNet C API Calls 103114 590.4710 487.3580 590.4710 51.5565
MXNet C API Concurrency 206227 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1804 0.8530 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9020 4.6520 0.0000 0.0230 0.0005
MXNDArraySetGradState 9020 4.6030 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1804 0.7740 0.0000 0.0010 0.0004
MXAutogradBackwardEx 902 498.6960 0.5250 0.9450 0.5529
MXNDArrayGetContext 14188 6.2530 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29882 768.6600 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3608 8816.8594 0.7010 3.7500 2.4437
FullyConnected 6132 592.6330 0.0080 0.3430 0.0966
argmax 2044 40.5360 0.0050 0.0480 0.0198
_backward_pick 1804 28.6430 0.0130 0.0320 0.0159
_backward_mul_scalar 1804 7.8840 0.0030 0.0220 0.0044
DeleteVariable 76480 542.2420 0.0010 0.9010 0.0071
Flatten 2044 85.2020 0.0080 0.1120 0.0417
Activation 8176 379.9440 0.0030 1.9440 0.0465
SetValueOp 1804 11.7280 0.0050 0.0180 0.0065
pick 1804 14.1960 0.0060 0.0280 0.0079
Pooling 4088 5676.1279 0.0640 3.1720 1.3885
Convolution 4088 23192.6055 0.2260 13.2050 5.6733
CopyCPU2CPU 2044 11.6250 0.0030 0.0360 0.0057
log_softmax 1804 56.2940 0.0150 0.0550 0.0312
_backward_copy 1804 93.4730 0.0220 0.0760 0.0518
_backward_Convolution 3608 33363.4609 2.6240 12.5610 9.2471
_mul_scalar 1804 6.4330 0.0020 0.0230 0.0036
_backward_log_softmax 1804 30.8400 0.0090 0.0400 0.0171
mean 5652 108.2760 0.0080 0.0730 0.0192
_backward_mean 1804 19.7950 0.0080 0.1030 0.0110
_backward_FullyConnected 5412 1038.9960 0.0280 0.5940 0.1920
WaitForVar 3848 34.2470 0.0060 0.0280 0.0089
broadcast_equal 2044 10.2020 0.0030 0.0110 0.0050
sgd_update 18040 180.9110 0.0020 0.0790 0.0100
_backward_Activation 7216 533.1590 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70422 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6138 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 902 3584.2200 0.3060 73.5870 3.9736
MXNDArrayCreateEx 1023 35.3950 0.0310 0.0640 0.0346
MXNDArrayWaitAll 903 19594.7559 5.6130 25.0610 21.6996
MXNDArraySyncCopyToCPU 1926 15067.0498 0.0600 24.1980 7.8230
MXNDArrayGetDType 1926 0.6990 0.0000 0.0010 0.0004
MXNDArrayGetShape 4875 2.2680 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2046 17.9850 0.0070 0.0280 0.0088
MXNDArrayFree 22929 235.3550 0.0000 0.9140 0.0103
MXNet C API Calls 103221 590.5780 487.3580 590.5780 51.6100
MXNet C API Concurrency 206441 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1806 0.8530 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9030 4.6560 0.0000 0.0230 0.0005
MXNDArraySetGradState 9030 4.6070 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1806 0.7740 0.0000 0.0010 0.0004
MXAutogradBackwardEx 903 499.3020 0.5250 0.9450 0.5529
MXNDArrayGetContext 14202 6.2610 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29913 769.4220 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3612 8826.4980 0.7010 3.7500 2.4437
FullyConnected 6138 593.1530 0.0080 0.3430 0.0966
argmax 2046 40.5730 0.0050 0.0480 0.0198
_backward_pick 1806 28.6720 0.0130 0.0320 0.0159
_backward_mul_scalar 1806 7.8930 0.0030 0.0220 0.0044
DeleteVariable 76560 542.7710 0.0010 0.9010 0.0071
Flatten 2046 85.2800 0.0080 0.1120 0.0417
Activation 8184 380.2940 0.0030 1.9440 0.0465
SetValueOp 1806 11.7410 0.0050 0.0180 0.0065
pick 1806 14.2110 0.0060 0.0280 0.0079
Pooling 4092 5681.5610 0.0640 3.1720 1.3885
Convolution 4092 23214.7695 0.2260 13.2050 5.6732
CopyCPU2CPU 2046 11.6360 0.0030 0.0360 0.0057
log_softmax 1806 56.3520 0.0150 0.0550 0.0312
_backward_copy 1806 93.5750 0.0220 0.0760 0.0518
_backward_Convolution 3612 33400.3008 2.6240 12.5610 9.2470
_mul_scalar 1806 6.4400 0.0020 0.0230 0.0036
_backward_log_softmax 1806 30.8720 0.0090 0.0400 0.0171
mean 5658 108.3520 0.0080 0.0730 0.0192
_backward_mean 1806 19.8160 0.0080 0.1030 0.0110
_backward_FullyConnected 5418 1040.0640 0.0280 0.5940 0.1920
WaitForVar 3852 34.2850 0.0060 0.0280 0.0089
broadcast_equal 2046 10.2110 0.0030 0.0110 0.0050
sgd_update 18060 181.1120 0.0020 0.0790 0.0100
_backward_Activation 7224 533.7000 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70496 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6144 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 903 3588.1221 0.3060 73.5870 3.9736
MXNDArrayCreateEx 1024 35.4290 0.0310 0.0640 0.0346
MXNDArrayWaitAll 904 19616.4727 5.6130 25.0610 21.6996
MXNDArraySyncCopyToCPU 1928 15081.7471 0.0600 24.1980 7.8225
MXNDArrayGetDType 1928 0.7000 0.0000 0.0010 0.0004
MXNDArrayGetShape 4880 2.2700 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2048 18.0030 0.0070 0.0280 0.0088
MXNDArrayFree 22952 235.6030 0.0000 0.9140 0.0103
MXNet C API Calls 103328 590.6850 487.3580 590.6850 51.6635
MXNet C API Concurrency 206655 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1808 0.8540 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9040 4.6620 0.0000 0.0230 0.0005
MXNDArraySetGradState 9040 4.6110 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1808 0.7750 0.0000 0.0010 0.0004
MXAutogradBackwardEx 904 499.8350 0.5250 0.9450 0.5529
MXNDArrayGetContext 14216 6.2670 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29944 770.1910 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3616 8836.1582 0.7010 3.7500 2.4436
FullyConnected 6144 593.8920 0.0080 0.3430 0.0967
argmax 2048 40.6180 0.0050 0.0480 0.0198
_backward_pick 1808 28.7040 0.0130 0.0320 0.0159
_backward_mul_scalar 1808 7.9010 0.0030 0.0220 0.0044
DeleteVariable 76640 543.3430 0.0010 0.9010 0.0071
Flatten 2048 85.3590 0.0080 0.1120 0.0417
Activation 8192 380.6700 0.0030 1.9440 0.0465
SetValueOp 1808 11.7540 0.0050 0.0180 0.0065
pick 1808 14.2280 0.0060 0.0280 0.0079
Pooling 4096 5686.9150 0.0640 3.1720 1.3884
Convolution 4096 23236.9414 0.2260 13.2050 5.6731
CopyCPU2CPU 2048 11.6460 0.0030 0.0360 0.0057
log_softmax 1808 56.4250 0.0150 0.0550 0.0312
_backward_copy 1808 93.6760 0.0220 0.0760 0.0518
_backward_Convolution 3616 33437.0352 2.6240 12.5610 9.2470
_mul_scalar 1808 6.4470 0.0020 0.0230 0.0036
_backward_log_softmax 1808 30.9150 0.0090 0.0400 0.0171
mean 5664 108.4570 0.0080 0.0730 0.0191
_backward_mean 1808 19.8390 0.0080 0.1030 0.0110
_backward_FullyConnected 5424 1041.4780 0.0280 0.5940 0.1920
WaitForVar 3856 34.3210 0.0060 0.0280 0.0089
broadcast_equal 2048 10.2200 0.0030 0.0110 0.0050
sgd_update 18080 181.4140 0.0020 0.0790 0.0100
_backward_Activation 7232 534.2980 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70570 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6150 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 904 3591.5920 0.3060 73.5870 3.9730
MXNDArrayCreateEx 1025 35.4620 0.0310 0.0640 0.0346
MXNDArrayWaitAll 905 19638.3320 5.6130 25.0610 21.6998
MXNDArraySyncCopyToCPU 1930 15096.0684 0.0600 24.1980 7.8218
MXNDArrayGetDType 1930 0.7010 0.0000 0.0010 0.0004
MXNDArrayGetShape 4885 2.2710 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2050 18.0190 0.0070 0.0280 0.0088
MXNDArrayFree 22975 235.8310 0.0000 0.9140 0.0103
MXNet C API Calls 103435 590.7920 487.3580 590.7920 51.7170
MXNet C API Concurrency 206869 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1810 0.8560 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9050 4.6650 0.0000 0.0230 0.0005
MXNDArraySetGradState 9050 4.6180 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1810 0.7760 0.0000 0.0010 0.0004
MXAutogradBackwardEx 905 500.3640 0.5250 0.9450 0.5529
MXNDArrayGetContext 14230 6.2770 0.0000 0.0190 0.0004
MXImperativeInvokeEx 29975 770.9850 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3620 8845.7432 0.7010 3.7500 2.4436
FullyConnected 6150 594.4110 0.0080 0.3430 0.0967
argmax 2050 40.6560 0.0050 0.0480 0.0198
_backward_pick 1810 28.7350 0.0130 0.0320 0.0159
_backward_mul_scalar 1810 7.9100 0.0030 0.0220 0.0044
DeleteVariable 76720 543.8570 0.0010 0.9010 0.0071
Flatten 2050 85.4370 0.0080 0.1120 0.0417
Activation 8200 381.0210 0.0030 1.9440 0.0465
SetValueOp 1810 11.7670 0.0050 0.0180 0.0065
pick 1810 14.2810 0.0060 0.0280 0.0079
Pooling 4100 5692.2622 0.0640 3.1720 1.3884
Convolution 4100 23259.1133 0.2260 13.2050 5.6730
CopyCPU2CPU 2050 11.6570 0.0030 0.0360 0.0057
log_softmax 1810 56.4840 0.0150 0.0550 0.0312
_backward_copy 1810 93.7800 0.0220 0.0760 0.0518
_backward_Convolution 3620 33474.1406 2.6240 12.5610 9.2470
_mul_scalar 1810 6.4540 0.0020 0.0230 0.0036
_backward_log_softmax 1810 30.9470 0.0090 0.0400 0.0171
mean 5670 108.5710 0.0080 0.0730 0.0191
_backward_mean 1810 19.8610 0.0080 0.1030 0.0110
_backward_FullyConnected 5430 1042.5430 0.0280 0.5940 0.1920
WaitForVar 3860 34.3580 0.0060 0.0280 0.0089
broadcast_equal 2050 10.2290 0.0030 0.0110 0.0050
sgd_update 18100 181.6150 0.0020 0.0790 0.0100
_backward_Activation 7240 534.8570 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70644 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6156 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 905 3595.4851 0.3060 73.5870 3.9729
MXNDArrayCreateEx 1026 35.4960 0.0310 0.0640 0.0346
MXNDArrayWaitAll 906 19660.1758 5.6130 25.0610 21.7000
MXNDArraySyncCopyToCPU 1932 15110.4893 0.0600 24.1980 7.8212
MXNDArrayGetDType 1932 0.7020 0.0000 0.0010 0.0004
MXNDArrayGetShape 4890 2.2720 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2052 18.0350 0.0070 0.0280 0.0088
MXNDArrayFree 22998 236.0630 0.0000 0.9140 0.0103
MXNet C API Calls 103542 590.8990 487.3580 590.8990 51.7705
MXNet C API Concurrency 207083 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1812 0.8570 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9060 4.6710 0.0000 0.0230 0.0005
MXNDArraySetGradState 9060 4.6230 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1812 0.7780 0.0000 0.0010 0.0004
MXAutogradBackwardEx 906 500.8930 0.5250 0.9450 0.5529
MXNDArrayGetContext 14244 6.2870 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30006 771.7520 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3624 8855.5010 0.7010 3.7500 2.4436
FullyConnected 6156 594.9540 0.0080 0.3430 0.0966
argmax 2052 40.6930 0.0050 0.0480 0.0198
_backward_pick 1812 28.7660 0.0130 0.0320 0.0159
_backward_mul_scalar 1812 7.9180 0.0030 0.0220 0.0044
DeleteVariable 76800 544.3850 0.0010 0.9010 0.0071
Flatten 2052 85.5160 0.0080 0.1120 0.0417
Activation 8208 381.3740 0.0030 1.9440 0.0465
SetValueOp 1812 11.7820 0.0050 0.0180 0.0065
pick 1812 14.2970 0.0060 0.0280 0.0079
Pooling 4104 5697.6440 0.0640 3.1720 1.3883
Convolution 4104 23281.3223 0.2260 13.2050 5.6728
CopyCPU2CPU 2052 11.6670 0.0030 0.0360 0.0057
log_softmax 1812 56.5440 0.0150 0.0550 0.0312
_backward_copy 1812 93.8840 0.0220 0.0760 0.0518
_backward_Convolution 3624 33511.1211 2.6240 12.5610 9.2470
_mul_scalar 1812 6.4610 0.0020 0.0230 0.0036
_backward_log_softmax 1812 30.9800 0.0090 0.0400 0.0171
mean 5676 108.6820 0.0080 0.0730 0.0191
_backward_mean 1812 19.8830 0.0080 0.1030 0.0110
_backward_FullyConnected 5436 1043.6071 0.0280 0.5940 0.1920
WaitForVar 3864 34.3940 0.0060 0.0280 0.0089
broadcast_equal 2052 10.2380 0.0030 0.0110 0.0050
sgd_update 18120 181.8100 0.0020 0.0790 0.0100
_backward_Activation 7248 535.4200 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70718 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6162 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 906 3599.3779 0.3060 73.5870 3.9728
MXNDArrayCreateEx 1027 35.5290 0.0310 0.0640 0.0346
MXNDArrayWaitAll 907 19681.8770 5.6130 25.0610 21.7000
MXNDArraySyncCopyToCPU 1934 15124.8516 0.0600 24.1980 7.8205
MXNDArrayGetDType 1934 0.7040 0.0000 0.0010 0.0004
MXNDArrayGetShape 4895 2.2750 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2054 18.0530 0.0070 0.0280 0.0088
MXNDArrayFree 23021 236.2960 0.0000 0.9140 0.0103
MXNet C API Calls 103649 591.0060 487.3580 591.0060 51.8240
MXNet C API Concurrency 207297 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1814 0.8570 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9070 4.6710 0.0000 0.0230 0.0005
MXNDArraySetGradState 9070 4.6250 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1814 0.7790 0.0000 0.0010 0.0004
MXAutogradBackwardEx 907 501.4260 0.5250 0.9450 0.5528
MXNDArrayGetContext 14258 6.2920 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30037 772.5250 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3628 8865.1162 0.7010 3.7500 2.4435
FullyConnected 6162 595.4890 0.0080 0.3430 0.0966
argmax 2054 40.7330 0.0050 0.0480 0.0198
_backward_pick 1814 28.7980 0.0130 0.0320 0.0159
_backward_mul_scalar 1814 7.9270 0.0030 0.0220 0.0044
DeleteVariable 76880 544.9070 0.0010 0.9010 0.0071
Flatten 2054 85.5950 0.0080 0.1120 0.0417
Activation 8216 381.7260 0.0030 1.9440 0.0465
SetValueOp 1814 11.7950 0.0050 0.0180 0.0065
pick 1814 14.3120 0.0060 0.0280 0.0079
Pooling 4108 5702.9702 0.0640 3.1720 1.3883
Convolution 4108 23303.5371 0.2260 13.2050 5.6727
CopyCPU2CPU 2054 11.6780 0.0030 0.0360 0.0057
log_softmax 1814 56.6030 0.0150 0.0550 0.0312
_backward_copy 1814 93.9860 0.0220 0.0760 0.0518
_backward_Convolution 3628 33547.9062 2.6240 12.5610 9.2469
_mul_scalar 1814 6.4680 0.0020 0.0230 0.0036
_backward_log_softmax 1814 31.0130 0.0090 0.0400 0.0171
mean 5682 108.7580 0.0080 0.0730 0.0191
_backward_mean 1814 19.9040 0.0080 0.1030 0.0110
_backward_FullyConnected 5442 1044.6801 0.0280 0.5940 0.1920
WaitForVar 3868 34.4310 0.0060 0.0280 0.0089
broadcast_equal 2054 10.2480 0.0030 0.0110 0.0050
sgd_update 18140 182.0100 0.0020 0.0790 0.0100
_backward_Activation 7256 535.9840 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70792 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6168 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 907 3603.2700 0.3060 73.5870 3.9727
MXNDArrayCreateEx 1028 35.5630 0.0310 0.0640 0.0346
MXNDArrayWaitAll 908 19703.6074 5.6130 25.0610 21.7000
MXNDArraySyncCopyToCPU 1936 15139.3789 0.0600 24.1980 7.8199
MXNDArrayGetDType 1936 0.7050 0.0000 0.0010 0.0004
MXNDArrayGetShape 4900 2.2760 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2056 18.0700 0.0070 0.0280 0.0088
MXNDArrayFree 23044 236.5190 0.0000 0.9140 0.0103
MXNet C API Calls 103756 591.1130 487.3580 591.1130 51.8775
MXNet C API Concurrency 207511 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1816 0.8580 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9080 4.6790 0.0000 0.0230 0.0005
MXNDArraySetGradState 9080 4.6300 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1816 0.7800 0.0000 0.0010 0.0004
MXAutogradBackwardEx 908 501.9750 0.5250 0.9450 0.5528
MXNDArrayGetContext 14272 6.2980 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30068 773.2930 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3632 8874.7764 0.7010 3.7500 2.4435
FullyConnected 6168 596.2470 0.0080 0.3430 0.0967
argmax 2056 40.7780 0.0050 0.0480 0.0198
_backward_pick 1816 28.8320 0.0130 0.0320 0.0159
_backward_mul_scalar 1816 7.9360 0.0030 0.0220 0.0044
DeleteVariable 76960 545.4310 0.0010 0.9010 0.0071
Flatten 2056 85.6720 0.0080 0.1120 0.0417
Activation 8224 382.1020 0.0030 1.9440 0.0465
SetValueOp 1816 11.8090 0.0050 0.0180 0.0065
pick 1816 14.3290 0.0060 0.0280 0.0079
Pooling 4112 5708.3398 0.0640 3.1720 1.3882
Convolution 4112 23325.7539 0.2260 13.2050 5.6726
CopyCPU2CPU 2056 11.6880 0.0030 0.0360 0.0057
log_softmax 1816 56.6740 0.0150 0.0550 0.0312
_backward_copy 1816 94.0860 0.0220 0.0760 0.0518
_backward_Convolution 3632 33584.6836 2.6240 12.5610 9.2469
_mul_scalar 1816 6.4750 0.0020 0.0230 0.0036
_backward_log_softmax 1816 31.0550 0.0090 0.0400 0.0171
mean 5688 108.8650 0.0080 0.0730 0.0191
_backward_mean 1816 19.9250 0.0080 0.1030 0.0110
_backward_FullyConnected 5448 1046.0900 0.0280 0.5940 0.1920
WaitForVar 3872 34.4670 0.0060 0.0280 0.0089
broadcast_equal 2056 10.2570 0.0030 0.0110 0.0050
sgd_update 18160 182.2850 0.0020 0.0790 0.0100
_backward_Activation 7264 536.5830 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70866 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6174 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 908 3607.1750 0.3060 73.5870 3.9727
MXNDArrayCreateEx 1029 35.5960 0.0310 0.0640 0.0346
MXNDArrayWaitAll 909 19725.4473 5.6130 25.0610 21.7002
MXNDArraySyncCopyToCPU 1938 15153.6689 0.0600 24.1980 7.8192
MXNDArrayGetDType 1938 0.7050 0.0000 0.0010 0.0004
MXNDArrayGetShape 4905 2.2790 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2058 18.0890 0.0070 0.0280 0.0088
MXNDArrayFree 23067 236.7520 0.0000 0.9140 0.0103
MXNet C API Calls 103863 591.2200 487.3580 591.2200 51.9310
MXNet C API Concurrency 207725 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1818 0.8590 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9090 4.6820 0.0000 0.0230 0.0005
MXNDArraySetGradState 9090 4.6320 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1818 0.7800 0.0000 0.0010 0.0004
MXAutogradBackwardEx 909 502.5080 0.5250 0.9450 0.5528
MXNDArrayGetContext 14286 6.3060 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30099 774.0740 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3636 8884.5557 0.7010 3.7500 2.4435
FullyConnected 6174 596.7670 0.0080 0.3430 0.0967
argmax 2058 40.8170 0.0050 0.0480 0.0198
_backward_pick 1818 28.8650 0.0130 0.0320 0.0159
_backward_mul_scalar 1818 7.9450 0.0030 0.0220 0.0044
DeleteVariable 77040 545.9560 0.0010 0.9010 0.0071
Flatten 2058 85.7510 0.0080 0.1120 0.0417
Activation 8232 382.4550 0.0030 1.9440 0.0465
SetValueOp 1818 11.8200 0.0050 0.0180 0.0065
pick 1818 14.3450 0.0060 0.0280 0.0079
Pooling 4116 5713.7202 0.0640 3.1720 1.3882
Convolution 4116 23347.8906 0.2260 13.2050 5.6725
CopyCPU2CPU 2058 11.6980 0.0030 0.0360 0.0057
log_softmax 1818 56.7340 0.0150 0.0550 0.0312
_backward_copy 1818 94.1900 0.0220 0.0760 0.0518
_backward_Convolution 3636 33621.6875 2.6240 12.5610 9.2469
_mul_scalar 1818 6.4820 0.0020 0.0230 0.0036
_backward_log_softmax 1818 31.0870 0.0090 0.0400 0.0171
mean 5694 108.9780 0.0080 0.0730 0.0191
_backward_mean 1818 19.9470 0.0080 0.1030 0.0110
_backward_FullyConnected 5454 1047.1639 0.0280 0.5940 0.1920
WaitForVar 3876 34.5030 0.0060 0.0280 0.0089
broadcast_equal 2058 10.2670 0.0030 0.0110 0.0050
sgd_update 18180 182.4870 0.0020 0.0790 0.0100
_backward_Activation 7272 537.1350 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 70940 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6180 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 909 3611.0869 0.3060 73.5870 3.9726
MXNDArrayCreateEx 1030 35.6290 0.0310 0.0640 0.0346
MXNDArrayWaitAll 910 19747.3848 5.6130 25.0610 21.7004
MXNDArraySyncCopyToCPU 1940 15168.0322 0.0600 24.1980 7.8186
MXNDArrayGetDType 1940 0.7060 0.0000 0.0010 0.0004
MXNDArrayGetShape 4910 2.2820 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2060 18.1060 0.0070 0.0280 0.0088
MXNDArrayFree 23090 236.9830 0.0000 0.9140 0.0103
MXNet C API Calls 103970 591.3270 487.3580 591.3270 51.9845
MXNet C API Concurrency 207939 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1820 0.8610 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9100 4.6880 0.0000 0.0230 0.0005
MXNDArraySetGradState 9100 4.6360 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1820 0.7800 0.0000 0.0010 0.0004
MXAutogradBackwardEx 910 503.0400 0.5250 0.9450 0.5528
MXNDArrayGetContext 14300 6.3100 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30130 774.8480 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3640 8894.2197 0.7010 3.7500 2.4435
FullyConnected 6180 597.3210 0.0080 0.3430 0.0967
argmax 2060 40.8540 0.0050 0.0480 0.0198
_backward_pick 1820 28.8960 0.0130 0.0320 0.0159
_backward_mul_scalar 1820 7.9540 0.0030 0.0220 0.0044
DeleteVariable 77120 546.4960 0.0010 0.9010 0.0071
Flatten 2060 85.8280 0.0080 0.1120 0.0417
Activation 8240 382.8070 0.0030 1.9440 0.0465
SetValueOp 1820 11.8330 0.0050 0.0180 0.0065
pick 1820 14.3620 0.0060 0.0280 0.0079
Pooling 4120 5719.0811 0.0640 3.1720 1.3881
Convolution 4120 23370.1250 0.2260 13.2050 5.6724
CopyCPU2CPU 2060 11.7080 0.0030 0.0360 0.0057
log_softmax 1820 56.7940 0.0150 0.0550 0.0312
_backward_copy 1820 94.2940 0.0220 0.0760 0.0518
_backward_Convolution 3640 33658.8828 2.6240 12.5610 9.2469
_mul_scalar 1820 6.4900 0.0020 0.0230 0.0036
_backward_log_softmax 1820 31.1190 0.0090 0.0400 0.0171
mean 5700 109.0900 0.0080 0.0730 0.0191
_backward_mean 1820 19.9680 0.0080 0.1030 0.0110
_backward_FullyConnected 5460 1048.2260 0.0280 0.5940 0.1920
WaitForVar 3880 34.5390 0.0060 0.0280 0.0089
broadcast_equal 2060 10.2770 0.0030 0.0110 0.0050
sgd_update 18200 182.6330 0.0020 0.0790 0.0100
_backward_Activation 7280 537.7460 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71014 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6186 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 910 3614.9810 0.3060 73.5870 3.9725
MXNDArrayCreateEx 1031 35.6620 0.0310 0.0640 0.0346
MXNDArrayWaitAll 911 19769.0547 5.6130 25.0610 21.7004
MXNDArraySyncCopyToCPU 1942 15182.4160 0.0600 24.1980 7.8179
MXNDArrayGetDType 1942 0.7060 0.0000 0.0010 0.0004
MXNDArrayGetShape 4915 2.2840 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2062 18.1220 0.0070 0.0280 0.0088
MXNDArrayFree 23113 237.2100 0.0000 0.9140 0.0103
MXNet C API Calls 104077 591.4340 487.3580 591.4340 52.0380
MXNet C API Concurrency 208153 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1822 0.8620 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9110 4.6930 0.0000 0.0230 0.0005
MXNDArraySetGradState 9110 4.6420 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1822 0.7820 0.0000 0.0010 0.0004
MXAutogradBackwardEx 911 503.5720 0.5250 0.9450 0.5528
MXNDArrayGetContext 14314 6.3160 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30161 775.6180 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3644 8903.8438 0.7010 3.7500 2.4434
FullyConnected 6186 597.8380 0.0080 0.3430 0.0966
argmax 2062 40.8920 0.0050 0.0480 0.0198
_backward_pick 1822 28.9260 0.0130 0.0320 0.0159
_backward_mul_scalar 1822 7.9630 0.0030 0.0220 0.0044
DeleteVariable 77200 547.0310 0.0010 0.9010 0.0071
Flatten 2062 85.9060 0.0080 0.1120 0.0417
Activation 8248 383.1610 0.0030 1.9440 0.0465
SetValueOp 1822 11.8470 0.0050 0.0180 0.0065
pick 1822 14.3770 0.0060 0.0280 0.0079
Pooling 4124 5724.4570 0.0640 3.1720 1.3881
Convolution 4124 23392.2969 0.2260 13.2050 5.6722
CopyCPU2CPU 2062 11.7180 0.0030 0.0360 0.0057
log_softmax 1822 56.8540 0.0150 0.0550 0.0312
_backward_copy 1822 94.3980 0.0220 0.0760 0.0518
_backward_Convolution 3644 33695.6484 2.6240 12.5610 9.2469
_mul_scalar 1822 6.4970 0.0020 0.0230 0.0036
_backward_log_softmax 1822 31.1510 0.0090 0.0400 0.0171
mean 5706 109.2020 0.0080 0.0730 0.0191
_backward_mean 1822 19.9890 0.0080 0.1030 0.0110
_backward_FullyConnected 5466 1049.2930 0.0280 0.5940 0.1920
WaitForVar 3884 34.5750 0.0060 0.0280 0.0089
broadcast_equal 2062 10.2870 0.0030 0.0110 0.0050
sgd_update 18220 182.8340 0.0020 0.0790 0.0100
_backward_Activation 7288 538.3100 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71088 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6192 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 911 3618.8799 0.3060 73.5870 3.9724
MXNDArrayCreateEx 1032 35.6950 0.0310 0.0640 0.0346
MXNDArrayWaitAll 912 19790.7520 5.6130 25.0610 21.7004
MXNDArraySyncCopyToCPU 1944 15197.1787 0.0600 24.1980 7.8175
MXNDArrayGetDType 1944 0.7060 0.0000 0.0010 0.0004
MXNDArrayGetShape 4920 2.2870 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2064 18.1370 0.0070 0.0280 0.0088
MXNDArrayFree 23136 237.4350 0.0000 0.9140 0.0103
MXNet C API Calls 104184 591.5410 487.3580 591.5410 52.0915
MXNet C API Concurrency 208367 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1824 0.8620 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9120 4.7000 0.0000 0.0230 0.0005
MXNDArraySetGradState 9120 4.6460 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1824 0.7840 0.0000 0.0010 0.0004
MXAutogradBackwardEx 912 504.1020 0.5250 0.9450 0.5527
MXNDArrayGetContext 14328 6.3220 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30192 776.3840 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3648 8913.5244 0.7010 3.7500 2.4434
FullyConnected 6192 598.5620 0.0080 0.3430 0.0967
argmax 2064 40.9380 0.0050 0.0480 0.0198
_backward_pick 1824 28.9580 0.0130 0.0320 0.0159
_backward_mul_scalar 1824 7.9720 0.0030 0.0220 0.0044
DeleteVariable 77280 547.5430 0.0010 0.9010 0.0071
Flatten 2064 85.9840 0.0080 0.1120 0.0417
Activation 8256 383.5450 0.0030 1.9440 0.0465
SetValueOp 1824 11.8600 0.0050 0.0180 0.0065
pick 1824 14.3930 0.0060 0.0280 0.0079
Pooling 4128 5729.8130 0.0640 3.1720 1.3880
Convolution 4128 23414.4922 0.2260 13.2050 5.6721
CopyCPU2CPU 2064 11.7280 0.0030 0.0360 0.0057
log_softmax 1824 56.9260 0.0150 0.0550 0.0312
_backward_copy 1824 94.4980 0.0220 0.0760 0.0518
_backward_Convolution 3648 33732.3984 2.6240 12.5610 9.2468
_mul_scalar 1824 6.5040 0.0020 0.0230 0.0036
_backward_log_softmax 1824 31.1940 0.0090 0.0400 0.0171
mean 5712 109.3050 0.0080 0.0730 0.0191
_backward_mean 1824 20.0100 0.0080 0.1030 0.0110
_backward_FullyConnected 5472 1050.7500 0.0280 0.5940 0.1920
WaitForVar 3888 34.6100 0.0060 0.0280 0.0089
broadcast_equal 2064 10.2970 0.0030 0.0110 0.0050
sgd_update 18240 183.1020 0.0020 0.0790 0.0100
_backward_Activation 7296 538.9010 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71162 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6198 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 912 3622.7881 0.3060 73.5870 3.9724
MXNDArrayCreateEx 1033 35.7280 0.0310 0.0640 0.0346
MXNDArrayWaitAll 913 19812.5586 5.6130 25.0610 21.7005
MXNDArraySyncCopyToCPU 1946 15211.4736 0.0600 24.1980 7.8168
MXNDArrayGetDType 1946 0.7060 0.0000 0.0010 0.0004
MXNDArrayGetShape 4925 2.2900 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2066 18.1540 0.0070 0.0280 0.0088
MXNDArrayFree 23159 237.6620 0.0000 0.9140 0.0103
MXNet C API Calls 104291 591.6480 487.3580 591.6480 52.1450
MXNet C API Concurrency 208581 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1826 0.8630 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9130 4.7040 0.0000 0.0230 0.0005
MXNDArraySetGradState 9130 4.6510 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1826 0.7860 0.0000 0.0010 0.0004
MXAutogradBackwardEx 913 504.6580 0.5250 0.9450 0.5527
MXNDArrayGetContext 14342 6.3290 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30223 777.1550 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3652 8923.0312 0.7010 3.7500 2.4433
FullyConnected 6198 599.1040 0.0080 0.3430 0.0967
argmax 2066 40.9760 0.0050 0.0480 0.0198
_backward_pick 1826 28.9900 0.0130 0.0320 0.0159
_backward_mul_scalar 1826 7.9800 0.0030 0.0220 0.0044
DeleteVariable 77360 548.0560 0.0010 0.9010 0.0071
Flatten 2066 86.0610 0.0080 0.1120 0.0417
Activation 8264 383.8970 0.0030 1.9440 0.0465
SetValueOp 1826 11.8730 0.0050 0.0180 0.0065
pick 1826 14.4070 0.0060 0.0280 0.0079
Pooling 4132 5735.1899 0.0640 3.1720 1.3880
Convolution 4132 23436.6445 0.2260 13.2050 5.6720
CopyCPU2CPU 2066 11.7380 0.0030 0.0360 0.0057
log_softmax 1826 56.9850 0.0150 0.0550 0.0312
_backward_copy 1826 94.6020 0.0220 0.0760 0.0518
_backward_Convolution 3652 33769.4492 2.6240 12.5610 9.2468
_mul_scalar 1826 6.5110 0.0020 0.0230 0.0036
_backward_log_softmax 1826 31.2270 0.0090 0.0400 0.0171
mean 5718 109.3790 0.0080 0.0730 0.0191
_backward_mean 1826 20.0310 0.0080 0.1030 0.0110
_backward_FullyConnected 5478 1051.8170 0.0280 0.5940 0.1920
WaitForVar 3892 34.6470 0.0060 0.0280 0.0089
broadcast_equal 2066 10.3070 0.0030 0.0110 0.0050
sgd_update 18260 183.2510 0.0020 0.0790 0.0100
_backward_Activation 7304 539.5260 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71236 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6204 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 913 3626.6870 0.3060 73.5870 3.9723
MXNDArrayCreateEx 1034 35.7630 0.0310 0.0640 0.0346
MXNDArrayWaitAll 914 19834.4023 5.6130 25.0610 21.7007
MXNDArraySyncCopyToCPU 1948 15225.8877 0.0600 24.1980 7.8162
MXNDArrayGetDType 1948 0.7070 0.0000 0.0010 0.0004
MXNDArrayGetShape 4930 2.2910 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2068 18.1720 0.0070 0.0280 0.0088
MXNDArrayFree 23182 237.8910 0.0000 0.9140 0.0103
MXNet C API Calls 104398 591.7550 487.3580 591.7550 52.1985
MXNet C API Concurrency 208795 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1828 0.8630 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9140 4.7080 0.0000 0.0230 0.0005
MXNDArraySetGradState 9140 4.6580 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1828 0.7870 0.0000 0.0010 0.0004
MXAutogradBackwardEx 914 505.1940 0.5250 0.9450 0.5527
MXNDArrayGetContext 14356 6.3350 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30254 777.9290 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3656 8932.7637 0.7010 3.7500 2.4433
FullyConnected 6204 599.6280 0.0080 0.3430 0.0967
argmax 2068 41.0140 0.0050 0.0480 0.0198
_backward_pick 1828 29.0200 0.0130 0.0320 0.0159
_backward_mul_scalar 1828 7.9890 0.0030 0.0220 0.0044
DeleteVariable 77440 548.5750 0.0010 0.9010 0.0071
Flatten 2068 86.1400 0.0080 0.1120 0.0417
Activation 8272 384.2490 0.0030 1.9440 0.0465
SetValueOp 1828 11.8860 0.0050 0.0180 0.0065
pick 1828 14.4220 0.0060 0.0280 0.0079
Pooling 4136 5740.5708 0.0640 3.1720 1.3880
Convolution 4136 23458.8750 0.2260 13.2050 5.6719
CopyCPU2CPU 2068 11.7470 0.0030 0.0360 0.0057
log_softmax 1828 57.0450 0.0150 0.0550 0.0312
_backward_copy 1828 94.7060 0.0220 0.0760 0.0518
_backward_Convolution 3656 33806.4844 2.6240 12.5610 9.2469
_mul_scalar 1828 6.5180 0.0020 0.0230 0.0036
_backward_log_softmax 1828 31.2580 0.0090 0.0400 0.0171
mean 5724 109.4900 0.0080 0.0730 0.0191
_backward_mean 1828 20.0520 0.0080 0.1030 0.0110
_backward_FullyConnected 5484 1052.8920 0.0280 0.5940 0.1920
WaitForVar 3896 34.6840 0.0060 0.0280 0.0089
broadcast_equal 2068 10.3160 0.0030 0.0110 0.0050
sgd_update 18280 183.4610 0.0020 0.0790 0.0100
_backward_Activation 7312 540.0760 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71310 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6210 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 914 3630.5940 0.3060 73.5870 3.9722
MXNDArrayCreateEx 1035 35.7960 0.0310 0.0640 0.0346
MXNDArrayWaitAll 915 19856.0664 5.6130 25.0610 21.7006
MXNDArraySyncCopyToCPU 1950 15240.2939 0.0600 24.1980 7.8155
MXNDArrayGetDType 1950 0.7090 0.0000 0.0010 0.0004
MXNDArrayGetShape 4935 2.2930 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2070 18.1890 0.0070 0.0280 0.0088
MXNDArrayFree 23205 238.1190 0.0000 0.9140 0.0103
MXNet C API Calls 104505 591.8620 487.3580 591.8620 52.2520
MXNet C API Concurrency 209009 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1830 0.8640 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9150 4.7140 0.0000 0.0230 0.0005
MXNDArraySetGradState 9150 4.6630 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1830 0.7880 0.0000 0.0010 0.0004
MXAutogradBackwardEx 915 505.7220 0.5250 0.9450 0.5527
MXNDArrayGetContext 14370 6.3440 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30285 778.6970 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3660 8942.3535 0.7010 3.7500 2.4433
FullyConnected 6210 600.1690 0.0080 0.3430 0.0966
argmax 2070 41.0510 0.0050 0.0480 0.0198
_backward_pick 1830 29.0510 0.0130 0.0320 0.0159
_backward_mul_scalar 1830 7.9990 0.0030 0.0220 0.0044
DeleteVariable 77520 549.1080 0.0010 0.9010 0.0071
Flatten 2070 86.2200 0.0080 0.1120 0.0417
Activation 8280 384.6030 0.0030 1.9440 0.0464
SetValueOp 1830 11.8990 0.0050 0.0180 0.0065
pick 1830 14.4370 0.0060 0.0280 0.0079
Pooling 4140 5745.9268 0.0640 3.1720 1.3879
Convolution 4140 23481.0605 0.2260 13.2050 5.6718
CopyCPU2CPU 2070 11.7570 0.0030 0.0360 0.0057
log_softmax 1830 57.1040 0.0150 0.0550 0.0312
_backward_copy 1830 94.8100 0.0220 0.0760 0.0518
_backward_Convolution 3660 33843.2500 2.6240 12.5610 9.2468
_mul_scalar 1830 6.5260 0.0020 0.0230 0.0036
_backward_log_softmax 1830 31.2900 0.0090 0.0400 0.0171
mean 5730 109.6070 0.0080 0.0730 0.0191
_backward_mean 1830 20.0730 0.0080 0.1030 0.0110
_backward_FullyConnected 5490 1053.9630 0.0280 0.5940 0.1920
WaitForVar 3900 34.7180 0.0060 0.0280 0.0089
broadcast_equal 2070 10.3260 0.0030 0.0110 0.0050
sgd_update 18300 183.6960 0.0020 0.0790 0.0100
_backward_Activation 7320 540.6330 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71384 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6216 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 915 3634.4900 0.3060 73.5870 3.9721
MXNDArrayCreateEx 1036 35.8300 0.0310 0.0640 0.0346
MXNDArrayWaitAll 916 19877.7734 5.6130 25.0610 21.7006
MXNDArraySyncCopyToCPU 1952 15255.0508 0.0600 24.1980 7.8151
MXNDArrayGetDType 1952 0.7100 0.0000 0.0010 0.0004
MXNDArrayGetShape 4940 2.2940 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2072 18.2060 0.0070 0.0280 0.0088
MXNDArrayFree 23228 238.3380 0.0000 0.9140 0.0103
MXNet C API Calls 104612 591.9690 487.3580 591.9690 52.3055
MXNet C API Concurrency 209223 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1832 0.8650 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9160 4.7220 0.0000 0.0230 0.0005
MXNDArraySetGradState 9160 4.6690 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1832 0.7890 0.0000 0.0010 0.0004
MXAutogradBackwardEx 916 506.2850 0.5250 0.9450 0.5527
MXNDArrayGetContext 14384 6.3480 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30316 779.4670 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3664 8952.1133 0.7010 3.7500 2.4433
FullyConnected 6216 600.9040 0.0080 0.3430 0.0967
argmax 2072 41.0950 0.0050 0.0480 0.0198
_backward_pick 1832 29.0830 0.0130 0.0320 0.0159
_backward_mul_scalar 1832 8.0080 0.0030 0.0220 0.0044
DeleteVariable 77600 549.6260 0.0010 0.9010 0.0071
Flatten 2072 86.2990 0.0080 0.1120 0.0417
Activation 8288 384.9820 0.0030 1.9440 0.0465
SetValueOp 1832 11.9120 0.0050 0.0180 0.0065
pick 1832 14.4530 0.0060 0.0280 0.0079
Pooling 4144 5751.3711 0.0640 3.1720 1.3879
Convolution 4144 23503.2129 0.2260 13.2050 5.6716
CopyCPU2CPU 2072 11.7680 0.0030 0.0360 0.0057
log_softmax 1832 57.1770 0.0150 0.0550 0.0312
_backward_copy 1832 94.9110 0.0220 0.0760 0.0518
_backward_Convolution 3664 33879.9805 2.6240 12.5610 9.2467
_mul_scalar 1832 6.5330 0.0020 0.0230 0.0036
_backward_log_softmax 1832 31.3320 0.0090 0.0400 0.0171
mean 5736 109.7100 0.0080 0.0730 0.0191
_backward_mean 1832 20.0950 0.0080 0.1030 0.0110
_backward_FullyConnected 5496 1055.3660 0.0280 0.5940 0.1920
WaitForVar 3904 34.7530 0.0060 0.0280 0.0089
broadcast_equal 2072 10.3350 0.0030 0.0110 0.0050
sgd_update 18320 183.9110 0.0020 0.0790 0.0100
_backward_Activation 7328 541.2980 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71458 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6222 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 916 3638.3921 0.3060 73.5870 3.9720
MXNDArrayCreateEx 1037 35.8630 0.0310 0.0640 0.0346
MXNDArrayWaitAll 917 19899.6172 5.6130 25.0610 21.7008
MXNDArraySyncCopyToCPU 1954 15269.4678 0.0600 24.1980 7.8145
MXNDArrayGetDType 1954 0.7100 0.0000 0.0010 0.0004
MXNDArrayGetShape 4945 2.2970 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2074 18.2230 0.0070 0.0280 0.0088
MXNDArrayFree 23251 238.5590 0.0000 0.9140 0.0103
MXNet C API Calls 104719 592.0760 487.3580 592.0760 52.3590
MXNet C API Concurrency 209437 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1834 0.8660 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9170 4.7270 0.0000 0.0230 0.0005
MXNDArraySetGradState 9170 4.6740 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1834 0.7900 0.0000 0.0010 0.0004
MXAutogradBackwardEx 917 506.8150 0.5250 0.9450 0.5527
MXNDArrayGetContext 14398 6.3530 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30347 780.2310 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3668 8961.8516 0.7010 3.7500 2.4433
FullyConnected 6222 601.4500 0.0080 0.3430 0.0967
argmax 2074 41.1320 0.0050 0.0480 0.0198
_backward_pick 1834 29.1140 0.0130 0.0320 0.0159
_backward_mul_scalar 1834 8.0170 0.0030 0.0220 0.0044
DeleteVariable 77680 550.1450 0.0010 0.9010 0.0071
Flatten 2074 86.3770 0.0080 0.1120 0.0416
Activation 8296 385.3350 0.0030 1.9440 0.0464
SetValueOp 1834 11.9250 0.0050 0.0180 0.0065
pick 1834 14.4690 0.0060 0.0280 0.0079
Pooling 4148 5756.7358 0.0640 3.1720 1.3878
Convolution 4148 23525.3652 0.2260 13.2050 5.6715
CopyCPU2CPU 2074 11.7780 0.0030 0.0360 0.0057
log_softmax 1834 57.2370 0.0150 0.0550 0.0312
_backward_copy 1834 95.0130 0.0220 0.0760 0.0518
_backward_Convolution 3668 33917.0234 2.6240 12.5610 9.2467
_mul_scalar 1834 6.5400 0.0020 0.0230 0.0036
_backward_log_softmax 1834 31.3650 0.0090 0.0400 0.0171
mean 5742 109.8220 0.0080 0.0730 0.0191
_backward_mean 1834 20.1180 0.0080 0.1030 0.0110
_backward_FullyConnected 5502 1056.4301 0.0280 0.5940 0.1920
WaitForVar 3908 34.7900 0.0060 0.0280 0.0089
broadcast_equal 2074 10.3440 0.0030 0.0110 0.0050
sgd_update 18340 184.1110 0.0020 0.0790 0.0100
_backward_Activation 7336 541.8730 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71532 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6228 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 917 3642.2920 0.3060 73.5870 3.9720
MXNDArrayCreateEx 1038 35.8970 0.0310 0.0640 0.0346
MXNDArrayWaitAll 918 19921.4863 5.6130 25.0610 21.7010
MXNDArraySyncCopyToCPU 1956 15283.8535 0.0600 24.1980 7.8138
MXNDArrayGetDType 1956 0.7110 0.0000 0.0010 0.0004
MXNDArrayGetShape 4950 2.3000 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2076 18.2400 0.0070 0.0280 0.0088
MXNDArrayFree 23274 238.7800 0.0000 0.9140 0.0103
MXNet C API Calls 104826 592.1830 487.3580 592.1830 52.4125
MXNet C API Concurrency 209651 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1836 0.8660 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9180 4.7340 0.0000 0.0230 0.0005
MXNDArraySetGradState 9180 4.6800 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1836 0.7910 0.0000 0.0010 0.0004
MXAutogradBackwardEx 918 507.3440 0.5250 0.9450 0.5527
MXNDArrayGetContext 14412 6.3610 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30378 781.0220 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3672 8971.5449 0.7010 3.7500 2.4432
FullyConnected 6228 601.9710 0.0080 0.3430 0.0967
argmax 2076 41.1700 0.0050 0.0480 0.0198
_backward_pick 1836 29.1450 0.0130 0.0320 0.0159
_backward_mul_scalar 1836 8.0260 0.0030 0.0220 0.0044
DeleteVariable 77760 550.6610 0.0010 0.9010 0.0071
Flatten 2076 86.4910 0.0080 0.1120 0.0417
Activation 8304 385.6860 0.0030 1.9440 0.0464
SetValueOp 1836 11.9370 0.0050 0.0180 0.0065
pick 1836 14.4850 0.0060 0.0280 0.0079
Pooling 4152 5762.0591 0.0640 3.1720 1.3878
Convolution 4152 23547.5859 0.2260 13.2050 5.6714
CopyCPU2CPU 2076 11.7880 0.0030 0.0360 0.0057
log_softmax 1836 57.2970 0.0150 0.0550 0.0312
_backward_copy 1836 95.1180 0.0220 0.0760 0.0518
_backward_Convolution 3672 33954.1055 2.6240 12.5610 9.2468
_mul_scalar 1836 6.5470 0.0020 0.0230 0.0036
_backward_log_softmax 1836 31.3970 0.0090 0.0400 0.0171
mean 5748 109.9350 0.0080 0.0730 0.0191
_backward_mean 1836 20.1390 0.0080 0.1030 0.0110
_backward_FullyConnected 5508 1057.5040 0.0280 0.5940 0.1920
WaitForVar 3912 34.8270 0.0060 0.0280 0.0089
broadcast_equal 2076 10.3540 0.0030 0.0110 0.0050
sgd_update 18360 184.2660 0.0020 0.0790 0.0100
_backward_Activation 7344 542.4940 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71606 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6234 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 918 3646.2070 0.3060 73.5870 3.9719
MXNDArrayCreateEx 1039 35.9300 0.0310 0.0640 0.0346
MXNDArrayWaitAll 919 19943.2266 5.6130 25.0610 21.7010
MXNDArraySyncCopyToCPU 1958 15298.2432 0.0600 24.1980 7.8132
MXNDArrayGetDType 1958 0.7120 0.0000 0.0010 0.0004
MXNDArrayGetShape 4955 2.3020 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2078 18.2580 0.0070 0.0280 0.0088
MXNDArrayFree 23297 239.0100 0.0000 0.9140 0.0103
MXNet C API Calls 104933 592.2900 487.3580 592.2900 52.4660
MXNet C API Concurrency 209865 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1838 0.8680 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9190 4.7400 0.0000 0.0230 0.0005
MXNDArraySetGradState 9190 4.6840 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1838 0.7930 0.0000 0.0010 0.0004
MXAutogradBackwardEx 919 507.8790 0.5250 0.9450 0.5526
MXNDArrayGetContext 14426 6.3720 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30409 781.8010 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3676 8981.3086 0.7010 3.7500 2.4432
FullyConnected 6234 602.5100 0.0080 0.3430 0.0966
argmax 2078 41.2070 0.0050 0.0480 0.0198
_backward_pick 1838 29.1760 0.0130 0.0320 0.0159
_backward_mul_scalar 1838 8.0350 0.0030 0.0220 0.0044
DeleteVariable 77840 551.2000 0.0010 0.9010 0.0071
Flatten 2078 86.5740 0.0080 0.1120 0.0417
Activation 8312 386.0400 0.0030 1.9440 0.0464
SetValueOp 1838 11.9500 0.0050 0.0180 0.0065
pick 1838 14.5000 0.0060 0.0280 0.0079
Pooling 4156 5767.4590 0.0640 3.1720 1.3877
Convolution 4156 23569.7578 0.2260 13.2050 5.6713
CopyCPU2CPU 2078 11.7980 0.0030 0.0360 0.0057
log_softmax 1838 57.3560 0.0150 0.0550 0.0312
_backward_copy 1838 95.2220 0.0220 0.0760 0.0518
_backward_Convolution 3676 33990.8672 2.6240 12.5610 9.2467
_mul_scalar 1838 6.5540 0.0020 0.0230 0.0036
_backward_log_softmax 1838 31.4290 0.0090 0.0400 0.0171
mean 5754 110.0470 0.0080 0.0730 0.0191
_backward_mean 1838 20.1610 0.0080 0.1030 0.0110
_backward_FullyConnected 5514 1058.5710 0.0280 0.5940 0.1920
WaitForVar 3916 34.8620 0.0060 0.0280 0.0089
broadcast_equal 2078 10.3640 0.0030 0.0110 0.0050
sgd_update 18380 184.4190 0.0020 0.0790 0.0100
_backward_Activation 7352 543.1220 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71680 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6240 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 919 3650.1101 0.3060 73.5870 3.9718
MXNDArrayCreateEx 1040 35.9640 0.0310 0.0640 0.0346
MXNDArrayWaitAll 920 19964.9258 5.6130 25.0610 21.7010
MXNDArraySyncCopyToCPU 1960 15312.9590 0.0600 24.1980 7.8127
MXNDArrayGetDType 1960 0.7130 0.0000 0.0010 0.0004
MXNDArrayGetShape 4960 2.3040 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2080 18.2750 0.0070 0.0280 0.0088
MXNDArrayFree 23320 239.2410 0.0000 0.9140 0.0103
MXNet C API Calls 105040 592.3970 487.3580 592.3970 52.5195
MXNet C API Concurrency 210079 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1840 0.8680 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9200 4.7460 0.0000 0.0230 0.0005
MXNDArraySetGradState 9200 4.6900 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1840 0.7930 0.0000 0.0010 0.0004
MXAutogradBackwardEx 920 508.4130 0.5250 0.9450 0.5526
MXNDArrayGetContext 14440 6.3790 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30440 782.5680 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3680 8990.9287 0.7010 3.7500 2.4432
FullyConnected 6240 603.2680 0.0080 0.3430 0.0967
argmax 2080 41.2520 0.0050 0.0480 0.0198
_backward_pick 1840 29.2080 0.0130 0.0320 0.0159
_backward_mul_scalar 1840 8.0430 0.0030 0.0220 0.0044
DeleteVariable 77920 551.7470 0.0010 0.9010 0.0071
Flatten 2080 86.6530 0.0080 0.1120 0.0417
Activation 8320 386.4190 0.0030 1.9440 0.0464
SetValueOp 1840 11.9620 0.0050 0.0180 0.0065
pick 1840 14.5170 0.0060 0.0280 0.0079
Pooling 4160 5772.7842 0.0640 3.1720 1.3877
Convolution 4160 23591.9766 0.2260 13.2050 5.6711
CopyCPU2CPU 2080 11.8080 0.0030 0.0360 0.0057
log_softmax 1840 57.4280 0.0150 0.0550 0.0312
_backward_copy 1840 95.3230 0.0220 0.0760 0.0518
_backward_Convolution 3680 34027.6602 2.6240 12.5610 9.2466
_mul_scalar 1840 6.5620 0.0020 0.0230 0.0036
_backward_log_softmax 1840 31.4720 0.0090 0.0400 0.0171
mean 5760 110.1530 0.0080 0.0730 0.0191
_backward_mean 1840 20.1820 0.0080 0.1030 0.0110
_backward_FullyConnected 5520 1059.9690 0.0280 0.5940 0.1920
WaitForVar 3920 34.8970 0.0060 0.0280 0.0089
broadcast_equal 2080 10.3740 0.0030 0.0110 0.0050
sgd_update 18400 184.6850 0.0020 0.0790 0.0100
_backward_Activation 7360 543.7010 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71754 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6246 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 920 3654.0171 0.3060 73.5870 3.9718
MXNDArrayCreateEx 1041 35.9980 0.0310 0.0640 0.0346
MXNDArrayWaitAll 921 19986.7910 5.6130 25.0610 21.7012
MXNDArraySyncCopyToCPU 1962 15327.3486 0.0600 24.1980 7.8121
MXNDArrayGetDType 1962 0.7130 0.0000 0.0010 0.0004
MXNDArrayGetShape 4965 2.3070 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2082 18.2920 0.0070 0.0280 0.0088
MXNDArrayFree 23343 239.4710 0.0000 0.9140 0.0103
MXNet C API Calls 105147 592.5040 487.3580 592.5040 52.5730
MXNet C API Concurrency 210293 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1842 0.8700 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9210 4.7500 0.0000 0.0230 0.0005
MXNDArraySetGradState 9210 4.6950 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1842 0.7940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 921 508.9470 0.5250 0.9450 0.5526
MXNDArrayGetContext 14454 6.3870 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30471 783.3360 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3684 9000.6533 0.7010 3.7500 2.4432
FullyConnected 6246 603.7920 0.0080 0.3430 0.0967
argmax 2082 41.2910 0.0050 0.0480 0.0198
_backward_pick 1842 29.2390 0.0130 0.0320 0.0159
_backward_mul_scalar 1842 8.0510 0.0030 0.0220 0.0044
DeleteVariable 78000 552.2720 0.0010 0.9010 0.0071
Flatten 2082 86.7320 0.0080 0.1120 0.0417
Activation 8328 386.7740 0.0030 1.9440 0.0464
SetValueOp 1842 11.9750 0.0050 0.0180 0.0065
pick 1842 14.5320 0.0060 0.0280 0.0079
Pooling 4164 5778.1660 0.0640 3.1720 1.3876
Convolution 4164 23614.1211 0.2260 13.2050 5.6710
CopyCPU2CPU 2082 11.8180 0.0030 0.0360 0.0057
log_softmax 1842 57.4880 0.0150 0.0550 0.0312
_backward_copy 1842 95.4260 0.0220 0.0760 0.0518
_backward_Convolution 3684 34064.7617 2.6240 12.5610 9.2467
_mul_scalar 1842 6.5690 0.0020 0.0230 0.0036
_backward_log_softmax 1842 31.5050 0.0090 0.0400 0.0171
mean 5766 110.2660 0.0080 0.0730 0.0191
_backward_mean 1842 20.2040 0.0080 0.1030 0.0110
_backward_FullyConnected 5526 1061.0410 0.0280 0.5940 0.1920
WaitForVar 3924 34.9320 0.0060 0.0280 0.0089
broadcast_equal 2082 10.3840 0.0030 0.0110 0.0050
sgd_update 18420 184.8320 0.0020 0.0790 0.0100
_backward_Activation 7368 544.3120 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71828 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6252 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 921 3657.9241 0.3060 73.5870 3.9717
MXNDArrayCreateEx 1042 36.0320 0.0310 0.0640 0.0346
MXNDArrayWaitAll 922 20008.5469 5.6130 25.0610 21.7012
MXNDArraySyncCopyToCPU 1964 15341.6465 0.0600 24.1980 7.8114
MXNDArrayGetDType 1964 0.7140 0.0000 0.0010 0.0004
MXNDArrayGetShape 4970 2.3090 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2084 18.3090 0.0070 0.0280 0.0088
MXNDArrayFree 23366 239.7060 0.0000 0.9140 0.0103
MXNet C API Calls 105254 592.6110 487.3580 592.6110 52.6265
MXNet C API Concurrency 210507 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1844 0.8700 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9220 4.7550 0.0000 0.0230 0.0005
MXNDArraySetGradState 9220 4.7000 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1844 0.7940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 922 509.4880 0.5250 0.9450 0.5526
MXNDArrayGetContext 14468 6.3940 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30502 784.1170 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3688 9010.0791 0.7010 3.7500 2.4431
FullyConnected 6252 604.3100 0.0080 0.3430 0.0967
argmax 2084 41.3290 0.0050 0.0480 0.0198
_backward_pick 1844 29.2690 0.0130 0.0320 0.0159
_backward_mul_scalar 1844 8.0600 0.0030 0.0220 0.0044
DeleteVariable 78080 552.8040 0.0010 0.9010 0.0071
Flatten 2084 86.8110 0.0080 0.1120 0.0417
Activation 8336 387.1250 0.0030 1.9440 0.0464
SetValueOp 1844 11.9880 0.0050 0.0180 0.0065
pick 1844 14.5480 0.0060 0.0280 0.0079
Pooling 4168 5783.5352 0.0640 3.1720 1.3876
Convolution 4168 23636.3555 0.2260 13.2050 5.6709
CopyCPU2CPU 2084 11.8280 0.0030 0.0360 0.0057
log_softmax 1844 57.5480 0.0150 0.0550 0.0312
_backward_copy 1844 95.5290 0.0220 0.0760 0.0518
_backward_Convolution 3688 34101.7891 2.6240 12.5610 9.2467
_mul_scalar 1844 6.5770 0.0020 0.0230 0.0036
_backward_log_softmax 1844 31.5370 0.0090 0.0400 0.0171
mean 5772 110.3430 0.0080 0.0730 0.0191
_backward_mean 1844 20.2260 0.0080 0.1030 0.0110
_backward_FullyConnected 5532 1062.1071 0.0280 0.5940 0.1920
WaitForVar 3928 34.9700 0.0060 0.0280 0.0089
broadcast_equal 2084 10.3930 0.0030 0.0110 0.0050
sgd_update 18440 185.0280 0.0020 0.0790 0.0100
_backward_Activation 7376 544.9100 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71902 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6258 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 922 3661.8279 0.3060 73.5870 3.9716
MXNDArrayCreateEx 1043 36.0660 0.0310 0.0640 0.0346
MXNDArrayWaitAll 923 20030.2812 5.6130 25.0610 21.7013
MXNDArraySyncCopyToCPU 1966 15356.0518 0.0600 24.1980 7.8108
MXNDArrayGetDType 1966 0.7150 0.0000 0.0010 0.0004
MXNDArrayGetShape 4975 2.3100 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2086 18.3270 0.0070 0.0280 0.0088
MXNDArrayFree 23389 239.9290 0.0000 0.9140 0.0103
MXNet C API Calls 105361 592.7180 487.3580 592.7180 52.6800
MXNet C API Concurrency 210721 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1846 0.8710 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9230 4.7610 0.0000 0.0230 0.0005
MXNDArraySetGradState 9230 4.7060 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1846 0.7940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 923 510.0200 0.5250 0.9450 0.5526
MXNDArrayGetContext 14482 6.4020 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30533 784.8890 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3692 9019.7451 0.7010 3.7500 2.4431
FullyConnected 6258 604.8550 0.0080 0.3430 0.0967
argmax 2086 41.3680 0.0050 0.0480 0.0198
_backward_pick 1846 29.3000 0.0130 0.0320 0.0159
_backward_mul_scalar 1846 8.0690 0.0030 0.0220 0.0044
DeleteVariable 78160 553.3160 0.0010 0.9010 0.0071
Flatten 2086 86.8890 0.0080 0.1120 0.0417
Activation 8344 387.4760 0.0030 1.9440 0.0464
SetValueOp 1846 12.0000 0.0050 0.0180 0.0065
pick 1846 14.5640 0.0060 0.0280 0.0079
Pooling 4172 5788.9199 0.0640 3.1720 1.3876
Convolution 4172 23658.5586 0.2260 13.2050 5.6708
CopyCPU2CPU 2086 11.8370 0.0030 0.0360 0.0057
log_softmax 1846 57.6070 0.0150 0.0550 0.0312
_backward_copy 1846 95.6330 0.0220 0.0760 0.0518
_backward_Convolution 3692 34138.6367 2.6240 12.5610 9.2467
_mul_scalar 1846 6.5830 0.0020 0.0230 0.0036
_backward_log_softmax 1846 31.5690 0.0090 0.0400 0.0171
mean 5778 110.4580 0.0080 0.0730 0.0191
_backward_mean 1846 20.2470 0.0080 0.1030 0.0110
_backward_FullyConnected 5538 1063.1780 0.0280 0.5940 0.1920
WaitForVar 3932 35.0060 0.0060 0.0280 0.0089
broadcast_equal 2086 10.4020 0.0030 0.0110 0.0050
sgd_update 18460 185.2280 0.0020 0.0790 0.0100
_backward_Activation 7384 545.4580 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 71976 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6264 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 923 3665.7490 0.3060 73.5870 3.9716
MXNDArrayCreateEx 1044 36.0990 0.0310 0.0640 0.0346
MXNDArrayWaitAll 924 20052.0039 5.6130 25.0610 21.7013
MXNDArraySyncCopyToCPU 1968 15370.7822 0.0600 24.1980 7.8104
MXNDArrayGetDType 1968 0.7160 0.0000 0.0010 0.0004
MXNDArrayGetShape 4980 2.3140 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2088 18.3450 0.0070 0.0280 0.0088
MXNDArrayFree 23412 240.1600 0.0000 0.9140 0.0103
MXNet C API Calls 105468 592.8250 487.3580 592.8250 52.7335
MXNet C API Concurrency 210935 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1848 0.8730 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9240 4.7670 0.0000 0.0230 0.0005
MXNDArraySetGradState 9240 4.7130 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1848 0.7940 0.0000 0.0010 0.0004
MXAutogradBackwardEx 924 510.5540 0.5250 0.9450 0.5525
MXNDArrayGetContext 14496 6.4080 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30564 785.6650 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3696 9029.4980 0.7010 3.7500 2.4430
FullyConnected 6264 605.5850 0.0080 0.3430 0.0967
argmax 2088 41.4130 0.0050 0.0480 0.0198
_backward_pick 1848 29.3330 0.0130 0.0320 0.0159
_backward_mul_scalar 1848 8.0780 0.0030 0.0220 0.0044
DeleteVariable 78240 553.8590 0.0010 0.9010 0.0071
Flatten 2088 86.9700 0.0080 0.1120 0.0417
Activation 8352 387.8530 0.0030 1.9440 0.0464
SetValueOp 1848 12.0130 0.0050 0.0180 0.0065
pick 1848 14.5810 0.0060 0.0280 0.0079
Pooling 4176 5794.3311 0.0640 3.1720 1.3875
Convolution 4176 23680.7246 0.2260 13.2050 5.6707
CopyCPU2CPU 2088 11.8470 0.0030 0.0360 0.0057
log_softmax 1848 57.6790 0.0150 0.0550 0.0312
_backward_copy 1848 95.7340 0.0220 0.0760 0.0518
_backward_Convolution 3696 34175.3789 2.6240 12.5610 9.2466
_mul_scalar 1848 6.5900 0.0020 0.0230 0.0036
_backward_log_softmax 1848 31.6110 0.0090 0.0400 0.0171
mean 5784 110.5670 0.0080 0.0730 0.0191
_backward_mean 1848 20.2690 0.0080 0.1030 0.0110
_backward_FullyConnected 5544 1064.5900 0.0280 0.5940 0.1920
WaitForVar 3936 35.0410 0.0060 0.0280 0.0089
broadcast_equal 2088 10.4110 0.0030 0.0110 0.0050
sgd_update 18480 185.5020 0.0020 0.0790 0.0100
_backward_Activation 7392 546.0430 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72050 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6270 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 924 3669.6440 0.3060 73.5870 3.9715
MXNDArrayCreateEx 1045 36.1310 0.0310 0.0640 0.0346
MXNDArrayWaitAll 925 20073.9199 5.6130 25.0610 21.7015
MXNDArraySyncCopyToCPU 1970 15385.1494 0.0600 24.1980 7.8097
MXNDArrayGetDType 1970 0.7160 0.0000 0.0010 0.0004
MXNDArrayGetShape 4985 2.3140 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2090 18.3620 0.0070 0.0280 0.0088
MXNDArrayFree 23435 240.3890 0.0000 0.9140 0.0103
MXNet C API Calls 105575 592.9320 487.3580 592.9320 52.7870
MXNet C API Concurrency 211149 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1850 0.8750 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9250 4.7690 0.0000 0.0230 0.0005
MXNDArraySetGradState 9250 4.7180 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1850 0.7950 0.0000 0.0010 0.0004
MXAutogradBackwardEx 925 511.0870 0.5250 0.9450 0.5525
MXNDArrayGetContext 14510 6.4160 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30595 786.4330 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3700 9039.2617 0.7010 3.7500 2.4430
FullyConnected 6270 606.1050 0.0080 0.3430 0.0967
argmax 2090 41.4520 0.0050 0.0480 0.0198
_backward_pick 1850 29.3640 0.0130 0.0320 0.0159
_backward_mul_scalar 1850 8.0860 0.0030 0.0220 0.0044
DeleteVariable 78320 554.3770 0.0010 0.9010 0.0071
Flatten 2090 87.0500 0.0080 0.1120 0.0417
Activation 8360 388.2070 0.0030 1.9440 0.0464
SetValueOp 1850 12.0260 0.0050 0.0180 0.0065
pick 1850 14.5960 0.0060 0.0280 0.0079
Pooling 4180 5799.6831 0.0640 3.1720 1.3875
Convolution 4180 23702.9199 0.2260 13.2050 5.6706
CopyCPU2CPU 2090 11.8560 0.0030 0.0360 0.0057
log_softmax 1850 57.7390 0.0150 0.0550 0.0312
_backward_copy 1850 95.8370 0.0220 0.0760 0.0518
_backward_Convolution 3700 34212.4727 2.6240 12.5610 9.2466
_mul_scalar 1850 6.5980 0.0020 0.0230 0.0036
_backward_log_softmax 1850 31.6810 0.0090 0.0400 0.0171
mean 5790 110.6820 0.0080 0.0730 0.0191
_backward_mean 1850 20.2900 0.0080 0.1030 0.0110
_backward_FullyConnected 5550 1065.6570 0.0280 0.5940 0.1920
WaitForVar 3940 35.0750 0.0060 0.0280 0.0089
broadcast_equal 2090 10.4200 0.0030 0.0110 0.0050
sgd_update 18500 185.7010 0.0020 0.0790 0.0100
_backward_Activation 7400 546.5880 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72124 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6276 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 925 3673.5471 0.3060 73.5870 3.9714
MXNDArrayCreateEx 1046 36.1640 0.0310 0.0640 0.0346
MXNDArrayWaitAll 926 20095.7871 5.6130 25.0610 21.7017
MXNDArraySyncCopyToCPU 1972 15399.5400 0.0600 24.1980 7.8091
MXNDArrayGetDType 1972 0.7170 0.0000 0.0010 0.0004
MXNDArrayGetShape 4990 2.3170 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2092 18.3790 0.0070 0.0280 0.0088
MXNDArrayFree 23458 240.6190 0.0000 0.9140 0.0103
MXNet C API Calls 105682 593.0390 487.3580 593.0390 52.8405
MXNet C API Concurrency 211363 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1852 0.8770 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9260 4.7750 0.0000 0.0230 0.0005
MXNDArraySetGradState 9260 4.7240 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1852 0.7960 0.0000 0.0010 0.0004
MXAutogradBackwardEx 926 511.6210 0.5250 0.9450 0.5525
MXNDArrayGetContext 14524 6.4190 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30626 787.2320 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3704 9048.9346 0.7010 3.7500 2.4430
FullyConnected 6276 606.6360 0.0080 0.3430 0.0967
argmax 2092 41.4920 0.0050 0.0480 0.0198
_backward_pick 1852 29.3940 0.0130 0.0320 0.0159
_backward_mul_scalar 1852 8.0950 0.0030 0.0220 0.0044
DeleteVariable 78400 554.9060 0.0010 0.9010 0.0071
Flatten 2092 87.1280 0.0080 0.1120 0.0416
Activation 8368 388.5580 0.0030 1.9440 0.0464
SetValueOp 1852 12.0390 0.0050 0.0180 0.0065
pick 1852 14.6110 0.0060 0.0280 0.0079
Pooling 4184 5805.0571 0.0640 3.1720 1.3874
Convolution 4184 23725.1348 0.2260 13.2050 5.6704
CopyCPU2CPU 2092 11.8660 0.0030 0.0360 0.0057
log_softmax 1852 57.7980 0.0150 0.0550 0.0312
_backward_copy 1852 95.9400 0.0220 0.0760 0.0518
_backward_Convolution 3704 34249.5781 2.6240 12.5610 9.2466
_mul_scalar 1852 6.6060 0.0020 0.0230 0.0036
_backward_log_softmax 1852 31.7140 0.0090 0.0400 0.0171
mean 5796 110.7960 0.0080 0.0730 0.0191
_backward_mean 1852 20.3120 0.0080 0.1030 0.0110
_backward_FullyConnected 5556 1066.7250 0.0280 0.5940 0.1920
WaitForVar 3944 35.1100 0.0060 0.0280 0.0089
broadcast_equal 2092 10.4300 0.0030 0.0110 0.0050
sgd_update 18520 185.9050 0.0020 0.0790 0.0100
_backward_Activation 7408 547.1350 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72198 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6282 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 926 3677.4551 0.3060 73.5870 3.9713
MXNDArrayCreateEx 1047 36.1970 0.0310 0.0640 0.0346
MXNDArrayWaitAll 927 20117.5762 5.6130 25.0610 21.7018
MXNDArraySyncCopyToCPU 1974 15413.9521 0.0600 24.1980 7.8085
MXNDArrayGetDType 1974 0.7170 0.0000 0.0010 0.0004
MXNDArrayGetShape 4995 2.3190 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2094 18.3960 0.0070 0.0280 0.0088
MXNDArrayFree 23481 240.8430 0.0000 0.9140 0.0103
MXNet C API Calls 105789 593.1460 487.3580 593.1460 52.8940
MXNet C API Concurrency 211577 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1854 0.8790 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9270 4.7790 0.0000 0.0230 0.0005
MXNDArraySetGradState 9270 4.7270 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1854 0.7970 0.0000 0.0010 0.0004
MXAutogradBackwardEx 927 512.1530 0.5250 0.9450 0.5525
MXNDArrayGetContext 14538 6.4240 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30657 788.0000 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3708 9058.7012 0.7010 3.7500 2.4430
FullyConnected 6282 607.1750 0.0080 0.3430 0.0967
argmax 2094 41.5300 0.0050 0.0480 0.0198
_backward_pick 1854 29.4250 0.0130 0.0320 0.0159
_backward_mul_scalar 1854 8.1040 0.0030 0.0220 0.0044
DeleteVariable 78480 555.4060 0.0010 0.9010 0.0071
Flatten 2094 87.2060 0.0080 0.1120 0.0416
Activation 8376 388.9100 0.0030 1.9440 0.0464
SetValueOp 1854 12.0530 0.0050 0.0180 0.0065
pick 1854 14.6270 0.0060 0.0280 0.0079
Pooling 4188 5810.4321 0.0640 3.1720 1.3874
Convolution 4188 23747.3047 0.2260 13.2050 5.6703
CopyCPU2CPU 2094 11.8760 0.0030 0.0360 0.0057
log_softmax 1854 57.8580 0.0150 0.0550 0.0312
_backward_copy 1854 96.0810 0.0220 0.0760 0.0518
_backward_Convolution 3708 34286.4297 2.6240 12.5610 9.2466
_mul_scalar 1854 6.6130 0.0020 0.0230 0.0036
_backward_log_softmax 1854 31.7450 0.0090 0.0400 0.0171
mean 5802 110.9160 0.0080 0.0730 0.0191
_backward_mean 1854 20.3330 0.0080 0.1030 0.0110
_backward_FullyConnected 5562 1067.7900 0.0280 0.5940 0.1920
WaitForVar 3948 35.1450 0.0060 0.0280 0.0089
broadcast_equal 2094 10.4390 0.0030 0.0110 0.0050
sgd_update 18540 186.1060 0.0020 0.0790 0.0100
_backward_Activation 7416 547.6800 0.0070 0.3900 0.0739
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72272 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6288 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 927 3681.3450 0.3060 73.5870 3.9712
MXNDArrayCreateEx 1048 36.2310 0.0310 0.0640 0.0346
MXNDArrayWaitAll 928 20139.2617 5.6130 25.0610 21.7018
MXNDArraySyncCopyToCPU 1976 15428.6436 0.0600 24.1980 7.8080
MXNDArrayGetDType 1976 0.7190 0.0000 0.0010 0.0004
MXNDArrayGetShape 5000 2.3230 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2096 18.4130 0.0070 0.0280 0.0088
MXNDArrayFree 23504 241.0650 0.0000 0.9140 0.0103
MXNet C API Calls 105896 593.2530 487.3580 593.2530 52.9475
MXNet C API Concurrency 211791 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1856 0.8800 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9280 4.7830 0.0000 0.0230 0.0005
MXNDArraySetGradState 9280 4.7290 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1856 0.7970 0.0000 0.0010 0.0004
MXAutogradBackwardEx 928 512.6840 0.5250 0.9450 0.5525
MXNDArrayGetContext 14552 6.4340 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30688 788.7740 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3712 9068.3369 0.7010 3.7500 2.4430
FullyConnected 6288 607.9290 0.0080 0.3430 0.0967
argmax 2096 41.5740 0.0050 0.0480 0.0198
_backward_pick 1856 29.4580 0.0130 0.0320 0.0159
_backward_mul_scalar 1856 8.1120 0.0030 0.0220 0.0044
DeleteVariable 78560 555.9280 0.0010 0.9010 0.0071
Flatten 2096 87.2850 0.0080 0.1120 0.0416
Activation 8384 389.2800 0.0030 1.9440 0.0464
SetValueOp 1856 12.0660 0.0050 0.0180 0.0065
pick 1856 14.6440 0.0060 0.0280 0.0079
Pooling 4192 5815.8418 0.0640 3.1720 1.3874
Convolution 4192 23769.5059 0.2260 13.2050 5.6702
CopyCPU2CPU 2096 11.8860 0.0030 0.0360 0.0057
log_softmax 1856 57.9310 0.0150 0.0550 0.0312
_backward_copy 1856 96.1820 0.0220 0.0760 0.0518
_backward_Convolution 3712 34323.1406 2.6240 12.5610 9.2465
_mul_scalar 1856 6.6200 0.0020 0.0230 0.0036
_backward_log_softmax 1856 31.7860 0.0090 0.0400 0.0171
mean 5808 110.9990 0.0080 0.0730 0.0191
_backward_mean 1856 20.3550 0.0080 0.1030 0.0110
_backward_FullyConnected 5568 1069.1960 0.0280 0.5940 0.1920
WaitForVar 3952 35.1830 0.0060 0.0280 0.0089
broadcast_equal 2096 10.4490 0.0030 0.0110 0.0050
sgd_update 18560 186.3800 0.0020 0.0790 0.0100
_backward_Activation 7424 548.2400 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72346 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6294 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 928 3685.2480 0.3060 73.5870 3.9712
MXNDArrayCreateEx 1049 36.2650 0.0310 0.0640 0.0346
MXNDArrayWaitAll 929 20161.1289 5.6130 25.0610 21.7020
MXNDArraySyncCopyToCPU 1978 15443.0430 0.0600 24.1980 7.8074
MXNDArrayGetDType 1978 0.7190 0.0000 0.0010 0.0004
MXNDArrayGetShape 5005 2.3260 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2098 18.4300 0.0070 0.0280 0.0088
MXNDArrayFree 23527 241.2910 0.0000 0.9140 0.0103
MXNet C API Calls 106003 593.3600 487.3580 593.3600 53.0010
MXNet C API Concurrency 212005 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1858 0.8820 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9290 4.7900 0.0000 0.0230 0.0005
MXNDArraySetGradState 9290 4.7310 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1858 0.7980 0.0000 0.0010 0.0004
MXAutogradBackwardEx 929 513.2160 0.5250 0.9450 0.5524
MXNDArrayGetContext 14566 6.4430 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30719 789.5450 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3716 9078.0020 0.7010 3.7500 2.4430
FullyConnected 6294 608.5320 0.0080 0.3430 0.0967
argmax 2098 41.6110 0.0050 0.0480 0.0198
_backward_pick 1858 29.4890 0.0130 0.0320 0.0159
_backward_mul_scalar 1858 8.1200 0.0030 0.0220 0.0044
DeleteVariable 78640 556.4660 0.0010 0.9010 0.0071
Flatten 2098 87.3650 0.0080 0.1120 0.0416
Activation 8392 389.6320 0.0030 1.9440 0.0464
SetValueOp 1858 12.0790 0.0050 0.0180 0.0065
pick 1858 14.6590 0.0060 0.0280 0.0079
Pooling 4196 5821.2432 0.0640 3.1720 1.3873
Convolution 4196 23791.6777 0.2260 13.2050 5.6701
CopyCPU2CPU 2098 11.8960 0.0030 0.0360 0.0057
log_softmax 1858 57.9900 0.0150 0.0550 0.0312
_backward_copy 1858 96.2870 0.0220 0.0760 0.0518
_backward_Convolution 3716 34360.2266 2.6240 12.5610 9.2466
_mul_scalar 1858 6.6270 0.0020 0.0230 0.0036
_backward_log_softmax 1858 31.8180 0.0090 0.0400 0.0171
mean 5814 111.1100 0.0080 0.0730 0.0191
_backward_mean 1858 20.3770 0.0080 0.1030 0.0110
_backward_FullyConnected 5574 1070.2660 0.0280 0.5940 0.1920
WaitForVar 3956 35.2200 0.0060 0.0280 0.0089
broadcast_equal 2098 10.4590 0.0030 0.0110 0.0050
sgd_update 18580 186.5800 0.0020 0.0790 0.0100
_backward_Activation 7432 548.7870 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72420 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6300 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 929 3689.1389 0.3060 73.5870 3.9711
MXNDArrayCreateEx 1050 36.2980 0.0310 0.0640 0.0346
MXNDArrayWaitAll 930 20182.9824 5.6130 25.0610 21.7021
MXNDArraySyncCopyToCPU 1980 15457.4463 0.0600 24.1980 7.8068
MXNDArrayGetDType 1980 0.7200 0.0000 0.0010 0.0004
MXNDArrayGetShape 5010 2.3270 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2100 18.4460 0.0070 0.0280 0.0088
MXNDArrayFree 23550 241.5200 0.0000 0.9140 0.0103
MXNet C API Calls 106110 593.4670 487.3580 593.4670 53.0545
MXNet C API Concurrency 212219 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1860 0.8820 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9300 4.7920 0.0000 0.0230 0.0005
MXNDArraySetGradState 9300 4.7360 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1860 0.8000 0.0000 0.0010 0.0004
MXAutogradBackwardEx 930 513.7490 0.5250 0.9450 0.5524
MXNDArrayGetContext 14580 6.4510 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30750 790.3140 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3720 9087.6562 0.7010 3.7500 2.4429
FullyConnected 6300 609.0490 0.0080 0.3430 0.0967
argmax 2100 41.6490 0.0050 0.0480 0.0198
_backward_pick 1860 29.5200 0.0130 0.0320 0.0159
_backward_mul_scalar 1860 8.1290 0.0030 0.0220 0.0044
DeleteVariable 78720 556.9980 0.0010 0.9010 0.0071
Flatten 2100 87.4450 0.0080 0.1120 0.0416
Activation 8400 389.9820 0.0030 1.9440 0.0464
SetValueOp 1860 12.0930 0.0050 0.0180 0.0065
pick 1860 14.6740 0.0060 0.0280 0.0079
Pooling 4200 5826.5908 0.0640 3.1720 1.3873
Convolution 4200 23813.9238 0.2260 13.2050 5.6700
CopyCPU2CPU 2100 11.9060 0.0030 0.0360 0.0057
log_softmax 1860 58.0500 0.0150 0.0550 0.0312
_backward_copy 1860 96.3920 0.0220 0.0760 0.0518
_backward_Convolution 3720 34397.2773 2.6240 12.5610 9.2466
_mul_scalar 1860 6.6340 0.0020 0.0230 0.0036
_backward_log_softmax 1860 31.8490 0.0090 0.0400 0.0171
mean 5820 111.2250 0.0080 0.0730 0.0191
_backward_mean 1860 20.3980 0.0080 0.1030 0.0110
_backward_FullyConnected 5580 1071.3730 0.0280 0.5940 0.1920
WaitForVar 3960 35.2540 0.0060 0.0280 0.0089
broadcast_equal 2100 10.4690 0.0030 0.0110 0.0050
sgd_update 18600 186.7290 0.0020 0.0790 0.0100
_backward_Activation 7440 549.4090 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72494 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6306 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 930 3693.0430 0.3060 73.5870 3.9710
MXNDArrayCreateEx 1051 36.3310 0.0310 0.0640 0.0346
MXNDArrayWaitAll 931 20204.6738 5.6130 25.0610 21.7021
MXNDArraySyncCopyToCPU 1982 15471.8369 0.0600 24.1980 7.8062
MXNDArrayGetDType 1982 0.7220 0.0000 0.0010 0.0004
MXNDArrayGetShape 5015 2.3310 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2102 18.4630 0.0070 0.0280 0.0088
MXNDArrayFree 23573 241.7490 0.0000 0.9140 0.0103
MXNet C API Calls 106217 593.5740 487.3580 593.5740 53.1080
MXNet C API Concurrency 212433 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1862 0.8820 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9310 4.7980 0.0000 0.0230 0.0005
MXNDArraySetGradState 9310 4.7400 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1862 0.8000 0.0000 0.0010 0.0004
MXAutogradBackwardEx 931 514.2810 0.5250 0.9450 0.5524
MXNDArrayGetContext 14594 6.4550 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30781 791.1080 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3724 9097.3154 0.7010 3.7500 2.4429
FullyConnected 6306 609.5720 0.0080 0.3430 0.0967
argmax 2102 41.6850 0.0050 0.0480 0.0198
_backward_pick 1862 29.5510 0.0130 0.0320 0.0159
_backward_mul_scalar 1862 8.1380 0.0030 0.0220 0.0044
DeleteVariable 78800 557.5320 0.0010 0.9010 0.0071
Flatten 2102 87.5230 0.0080 0.1120 0.0416
Activation 8408 390.3530 0.0030 1.9440 0.0464
SetValueOp 1862 12.1060 0.0050 0.0180 0.0065
pick 1862 14.6900 0.0060 0.0280 0.0079
Pooling 4204 5831.9678 0.0640 3.1720 1.3872
Convolution 4204 23836.0801 0.2260 13.2050 5.6699
CopyCPU2CPU 2102 11.9150 0.0030 0.0360 0.0057
log_softmax 1862 58.1110 0.0150 0.0550 0.0312
_backward_copy 1862 96.4960 0.0220 0.0760 0.0518
_backward_Convolution 3724 34434.0586 2.6240 12.5610 9.2465
_mul_scalar 1862 6.6410 0.0020 0.0230 0.0036
_backward_log_softmax 1862 31.8810 0.0090 0.0400 0.0171
mean 5826 111.3340 0.0080 0.0730 0.0191
_backward_mean 1862 20.4200 0.0080 0.1030 0.0110
_backward_FullyConnected 5586 1072.4390 0.0280 0.5940 0.1920
WaitForVar 3964 35.2900 0.0060 0.0280 0.0089
broadcast_equal 2102 10.4780 0.0030 0.0110 0.0050
sgd_update 18620 186.9310 0.0020 0.0790 0.0100
_backward_Activation 7448 549.9840 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72568 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6312 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 931 3696.9399 0.3060 73.5870 3.9709
MXNDArrayCreateEx 1052 36.3650 0.0310 0.0640 0.0346
MXNDArrayWaitAll 932 20226.3984 5.6130 25.0610 21.7021
MXNDArraySyncCopyToCPU 1984 15486.2129 0.0600 24.1980 7.8056
MXNDArrayGetDType 1984 0.7220 0.0000 0.0010 0.0004
MXNDArrayGetShape 5020 2.3350 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2104 18.4800 0.0070 0.0280 0.0088
MXNDArrayFree 23596 241.9700 0.0000 0.9140 0.0103
MXNet C API Calls 106324 593.6810 487.3580 593.6810 53.1615
MXNet C API Concurrency 212647 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1864 0.8840 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9320 4.8020 0.0000 0.0230 0.0005
MXNDArraySetGradState 9320 4.7430 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1864 0.8010 0.0000 0.0010 0.0004
MXAutogradBackwardEx 932 514.8160 0.5250 0.9450 0.5524
MXNDArrayGetContext 14608 6.4650 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30812 791.8840 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3728 9106.9824 0.7010 3.7500 2.4429
FullyConnected 6312 610.1160 0.0080 0.3430 0.0967
argmax 2104 41.7220 0.0050 0.0480 0.0198
_backward_pick 1864 29.5810 0.0130 0.0320 0.0159
_backward_mul_scalar 1864 8.1470 0.0030 0.0220 0.0044
DeleteVariable 78880 558.0650 0.0010 0.9010 0.0071
Flatten 2104 87.6010 0.0080 0.1120 0.0416
Activation 8416 390.7100 0.0030 1.9440 0.0464
SetValueOp 1864 12.1180 0.0050 0.0180 0.0065
pick 1864 14.7040 0.0060 0.0280 0.0079
Pooling 4208 5837.3218 0.0640 3.1720 1.3872
Convolution 4208 23858.2520 0.2260 13.2050 5.6697
CopyCPU2CPU 2104 11.9250 0.0030 0.0360 0.0057
log_softmax 1864 58.1700 0.0150 0.0550 0.0312
_backward_copy 1864 96.5990 0.0220 0.0760 0.0518
_backward_Convolution 3728 34470.8672 2.6240 12.5610 9.2465
_mul_scalar 1864 6.6480 0.0020 0.0230 0.0036
_backward_log_softmax 1864 31.9130 0.0090 0.0400 0.0171
mean 5832 111.4480 0.0080 0.0730 0.0191
_backward_mean 1864 20.4410 0.0080 0.1030 0.0110
_backward_FullyConnected 5592 1073.5081 0.0280 0.5940 0.1920
WaitForVar 3968 35.3260 0.0060 0.0280 0.0089
broadcast_equal 2104 10.4870 0.0030 0.0110 0.0050
sgd_update 18640 187.1290 0.0020 0.0790 0.0100
_backward_Activation 7456 550.5520 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72642 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6318 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 932 3700.8479 0.3060 73.5870 3.9709
MXNDArrayCreateEx 1053 36.3980 0.0310 0.0640 0.0346
MXNDArrayWaitAll 933 20248.2656 5.6130 25.0610 21.7023
MXNDArraySyncCopyToCPU 1986 15500.5508 0.0600 24.1980 7.8049
MXNDArrayGetDType 1986 0.7220 0.0000 0.0010 0.0004
MXNDArrayGetShape 5025 2.3370 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2106 18.4960 0.0070 0.0280 0.0088
MXNDArrayFree 23619 242.2170 0.0000 0.9140 0.0103
MXNet C API Calls 106431 593.7880 487.3580 593.7880 53.2150
MXNet C API Concurrency 212861 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1866 0.8840 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9330 4.8080 0.0000 0.0230 0.0005
MXNDArraySetGradState 9330 4.7470 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1866 0.8020 0.0000 0.0010 0.0004
MXAutogradBackwardEx 933 515.3430 0.5250 0.9450 0.5524
MXNDArrayGetContext 14622 6.4690 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30843 792.6470 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3732 9116.6826 0.7010 3.7500 2.4428
FullyConnected 6318 610.6340 0.0080 0.3430 0.0966
argmax 2106 41.7600 0.0050 0.0480 0.0198
_backward_pick 1866 29.6100 0.0130 0.0320 0.0159
_backward_mul_scalar 1866 8.1550 0.0030 0.0220 0.0044
DeleteVariable 78960 558.6170 0.0010 0.9010 0.0071
Flatten 2106 87.6790 0.0080 0.1120 0.0416
Activation 8424 391.0580 0.0030 1.9440 0.0464
SetValueOp 1866 12.1310 0.0050 0.0180 0.0065
pick 1866 14.7190 0.0060 0.0280 0.0079
Pooling 4212 5842.7002 0.0640 3.1720 1.3872
Convolution 4212 23880.3281 0.2260 13.2050 5.6696
CopyCPU2CPU 2106 11.9350 0.0030 0.0360 0.0057
log_softmax 1866 58.2300 0.0150 0.0550 0.0312
_backward_copy 1866 96.7020 0.0220 0.0760 0.0518
_backward_Convolution 3732 34507.9297 2.6240 12.5610 9.2465
_mul_scalar 1866 6.6550 0.0020 0.0230 0.0036
_backward_log_softmax 1866 31.9450 0.0090 0.0400 0.0171
mean 5838 111.5590 0.0080 0.0730 0.0191
_backward_mean 1866 20.4610 0.0080 0.1030 0.0110
_backward_FullyConnected 5598 1074.5690 0.0280 0.5940 0.1920
WaitForVar 3972 35.3600 0.0060 0.0280 0.0089
broadcast_equal 2106 10.4970 0.0030 0.0110 0.0050
sgd_update 18660 187.2790 0.0020 0.0790 0.0100
_backward_Activation 7464 551.1640 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72716 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6324 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 933 3704.7439 0.3060 73.5870 3.9708
MXNDArrayCreateEx 1054 36.4320 0.0310 0.0640 0.0346
MXNDArrayWaitAll 934 20270.1777 5.6130 25.0610 21.7025
MXNDArraySyncCopyToCPU 1988 15514.9473 0.0600 24.1980 7.8043
MXNDArrayGetDType 1988 0.7240 0.0000 0.0010 0.0004
MXNDArrayGetShape 5030 2.3390 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2108 18.5140 0.0070 0.0280 0.0088
MXNDArrayFree 23642 242.4370 0.0000 0.9140 0.0103
MXNet C API Calls 106538 593.8950 487.3580 593.8950 53.2685
MXNet C API Concurrency 213075 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1868 0.8850 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9340 4.8130 0.0000 0.0230 0.0005
MXNDArraySetGradState 9340 4.7510 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1868 0.8030 0.0000 0.0010 0.0004
MXAutogradBackwardEx 934 515.8700 0.5250 0.9450 0.5523
MXNDArrayGetContext 14636 6.4730 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30874 793.4150 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3736 9126.4180 0.7010 3.7500 2.4428
FullyConnected 6324 611.1600 0.0080 0.3430 0.0966
argmax 2108 41.7990 0.0050 0.0480 0.0198
_backward_pick 1868 29.6380 0.0130 0.0320 0.0159
_backward_mul_scalar 1868 8.1640 0.0030 0.0220 0.0044
DeleteVariable 79040 559.1510 0.0010 0.9010 0.0071
Flatten 2108 87.7590 0.0080 0.1120 0.0416
Activation 8432 391.4080 0.0030 1.9440 0.0464
SetValueOp 1868 12.1430 0.0050 0.0180 0.0065
pick 1868 14.7340 0.0060 0.0280 0.0079
Pooling 4216 5848.0718 0.0640 3.1720 1.3871
Convolution 4216 23902.5547 0.2260 13.2050 5.6695
CopyCPU2CPU 2108 11.9450 0.0030 0.0360 0.0057
log_softmax 1868 58.2890 0.0150 0.0550 0.0312
_backward_copy 1868 96.8010 0.0220 0.0760 0.0518
_backward_Convolution 3736 34545.0078 2.6240 12.5610 9.2465
_mul_scalar 1868 6.6620 0.0020 0.0230 0.0036
_backward_log_softmax 1868 31.9770 0.0090 0.0400 0.0171
mean 5844 111.6720 0.0080 0.0730 0.0191
_backward_mean 1868 20.4820 0.0080 0.1030 0.0110
_backward_FullyConnected 5604 1075.6331 0.0280 0.5940 0.1919
WaitForVar 3976 35.3940 0.0060 0.0280 0.0089
broadcast_equal 2108 10.5060 0.0030 0.0110 0.0050
sgd_update 18680 187.4810 0.0020 0.0790 0.0100
_backward_Activation 7472 551.7220 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72790 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6330 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 934 3708.6470 0.3060 73.5870 3.9707
MXNDArrayCreateEx 1055 36.4650 0.0310 0.0640 0.0346
MXNDArrayWaitAll 935 20291.9688 5.6130 25.0610 21.7026
MXNDArraySyncCopyToCPU 1990 15529.2734 0.0600 24.1980 7.8037
MXNDArrayGetDType 1990 0.7250 0.0000 0.0010 0.0004
MXNDArrayGetShape 5035 2.3400 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2110 18.5310 0.0070 0.0280 0.0088
MXNDArrayFree 23665 242.6560 0.0000 0.9140 0.0103
MXNet C API Calls 106645 594.0020 487.3580 594.0020 53.3220
MXNet C API Concurrency 213289 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1870 0.8850 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9350 4.8180 0.0000 0.0230 0.0005
MXNDArraySetGradState 9350 4.7530 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1870 0.8040 0.0000 0.0010 0.0004
MXAutogradBackwardEx 935 516.3930 0.5230 0.9450 0.5523
MXNDArrayGetContext 14650 6.4820 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30905 794.1800 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3740 9136.0420 0.7010 3.7500 2.4428
FullyConnected 6330 611.6730 0.0080 0.3430 0.0966
argmax 2110 41.8350 0.0050 0.0480 0.0198
_backward_pick 1870 29.6680 0.0130 0.0320 0.0159
_backward_mul_scalar 1870 8.1720 0.0030 0.0220 0.0044
DeleteVariable 79120 559.6550 0.0010 0.9010 0.0071
Flatten 2110 87.8360 0.0080 0.1120 0.0416
Activation 8440 391.7590 0.0030 1.9440 0.0464
SetValueOp 1870 12.1550 0.0050 0.0180 0.0065
pick 1870 14.7490 0.0060 0.0280 0.0079
Pooling 4220 5853.4268 0.0640 3.1720 1.3871
Convolution 4220 23924.7051 0.2260 13.2050 5.6694
CopyCPU2CPU 2110 11.9540 0.0030 0.0360 0.0057
log_softmax 1870 58.3470 0.0150 0.0550 0.0312
_backward_copy 1870 96.9030 0.0220 0.0760 0.0518
_backward_Convolution 3740 34581.9414 2.6240 12.5610 9.2465
_mul_scalar 1870 6.6680 0.0020 0.0230 0.0036
_backward_log_softmax 1870 32.0080 0.0090 0.0400 0.0171
mean 5850 111.7470 0.0080 0.0730 0.0191
_backward_mean 1870 20.5020 0.0080 0.1030 0.0110
_backward_FullyConnected 5610 1076.7340 0.0280 0.5940 0.1919
WaitForVar 3980 35.4290 0.0060 0.0280 0.0089
broadcast_equal 2110 10.5160 0.0030 0.0110 0.0050
sgd_update 18700 187.6810 0.0020 0.0790 0.0100
_backward_Activation 7480 552.2690 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72864 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6336 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 935 3712.5669 0.3060 73.5870 3.9707
MXNDArrayCreateEx 1056 36.4990 0.0310 0.0640 0.0346
MXNDArrayWaitAll 936 20313.6641 5.6130 25.0610 21.7026
MXNDArraySyncCopyToCPU 1992 15543.5225 0.0600 24.1980 7.8030
MXNDArrayGetDType 1992 0.7250 0.0000 0.0010 0.0004
MXNDArrayGetShape 5040 2.3420 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2112 18.5480 0.0070 0.0280 0.0088
MXNDArrayFree 23688 242.8740 0.0000 0.9140 0.0103
MXNet C API Calls 106752 594.1090 487.3580 594.1090 53.3755
MXNet C API Concurrency 213503 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1872 0.8860 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9360 4.8220 0.0000 0.0230 0.0005
MXNDArraySetGradState 9360 4.7600 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1872 0.8050 0.0000 0.0010 0.0004
MXAutogradBackwardEx 936 516.9180 0.5230 0.9450 0.5523
MXNDArrayGetContext 14664 6.4900 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30936 794.9490 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3744 9145.6299 0.7010 3.7500 2.4427
FullyConnected 6336 612.1880 0.0080 0.3430 0.0966
argmax 2112 41.8730 0.0050 0.0480 0.0198
_backward_pick 1872 29.6970 0.0130 0.0320 0.0159
_backward_mul_scalar 1872 8.1800 0.0030 0.0220 0.0044
DeleteVariable 79200 560.1590 0.0010 0.9010 0.0071
Flatten 2112 87.9140 0.0080 0.1120 0.0416
Activation 8448 392.1080 0.0030 1.9440 0.0464
SetValueOp 1872 12.1670 0.0050 0.0180 0.0065
pick 1872 14.7640 0.0060 0.0280 0.0079
Pooling 4224 5858.7959 0.0640 3.1720 1.3870
Convolution 4224 23946.7383 0.2260 13.2050 5.6692
CopyCPU2CPU 2112 11.9630 0.0030 0.0360 0.0057
log_softmax 1872 58.4380 0.0150 0.0550 0.0312
_backward_copy 1872 97.0060 0.0220 0.0760 0.0518
_backward_Convolution 3744 34618.6992 2.6240 12.5610 9.2464
_mul_scalar 1872 6.6750 0.0020 0.0230 0.0036
_backward_log_softmax 1872 32.0390 0.0090 0.0400 0.0171
mean 5856 111.8190 0.0080 0.0730 0.0191
_backward_mean 1872 20.5230 0.0080 0.1030 0.0110
_backward_FullyConnected 5616 1077.7960 0.0280 0.5940 0.1919
WaitForVar 3984 35.4650 0.0060 0.0280 0.0089
broadcast_equal 2112 10.5250 0.0030 0.0110 0.0050
sgd_update 18720 187.8310 0.0020 0.0790 0.0100
_backward_Activation 7488 552.8810 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 72938 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6342 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 936 3716.4890 0.3060 73.5870 3.9706
MXNDArrayCreateEx 1057 36.5320 0.0310 0.0640 0.0346
MXNDArrayWaitAll 937 20335.5488 5.6130 25.0610 21.7028
MXNDArraySyncCopyToCPU 1994 15557.7217 0.0600 24.1980 7.8023
MXNDArrayGetDType 1994 0.7250 0.0000 0.0010 0.0004
MXNDArrayGetShape 5045 2.3440 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2114 18.5650 0.0070 0.0280 0.0088
MXNDArrayFree 23711 243.0910 0.0000 0.9140 0.0103
MXNet C API Calls 106859 594.2160 487.3580 594.2160 53.4290
MXNet C API Concurrency 213717 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1874 0.8870 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9370 4.8280 0.0000 0.0230 0.0005
MXNDArraySetGradState 9370 4.7650 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1874 0.8050 0.0000 0.0010 0.0004
MXAutogradBackwardEx 937 517.4660 0.5230 0.9450 0.5523
MXNDArrayGetContext 14678 6.4970 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30967 795.7360 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3748 9155.3867 0.7010 3.7500 2.4427
FullyConnected 6342 612.7450 0.0080 0.3430 0.0966
argmax 2114 41.9090 0.0050 0.0480 0.0198
_backward_pick 1874 29.7280 0.0130 0.0320 0.0159
_backward_mul_scalar 1874 8.1880 0.0030 0.0220 0.0044
DeleteVariable 79280 560.6710 0.0010 0.9010 0.0071
Flatten 2114 87.9910 0.0080 0.1120 0.0416
Activation 8456 392.4570 0.0030 1.9440 0.0464
SetValueOp 1874 12.1780 0.0050 0.0180 0.0065
pick 1874 14.7790 0.0060 0.0280 0.0079
Pooling 4228 5864.1479 0.0640 3.1720 1.3870
Convolution 4228 23968.8789 0.2260 13.2050 5.6691
CopyCPU2CPU 2114 11.9730 0.0030 0.0360 0.0057
log_softmax 1874 58.4980 0.0150 0.0550 0.0312
_backward_copy 1874 97.1080 0.0220 0.0760 0.0518
_backward_Convolution 3748 34655.7227 2.6240 12.5610 9.2465
_mul_scalar 1874 6.6820 0.0020 0.0230 0.0036
_backward_log_softmax 1874 32.0700 0.0090 0.0400 0.0171
mean 5862 111.9350 0.0080 0.0730 0.0191
_backward_mean 1874 20.5440 0.0080 0.1030 0.0110
_backward_FullyConnected 5622 1078.8560 0.0280 0.5940 0.1919
WaitForVar 3988 35.4990 0.0060 0.0280 0.0089
broadcast_equal 2114 10.5340 0.0030 0.0110 0.0050
sgd_update 18740 188.0320 0.0020 0.0790 0.0100
_backward_Activation 7496 553.4400 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 73012 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6348 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 937 3720.3840 0.3060 73.5870 3.9705
MXNDArrayCreateEx 1058 36.5650 0.0310 0.0640 0.0346
MXNDArrayWaitAll 938 20357.4355 5.6130 25.0610 21.7030
MXNDArraySyncCopyToCPU 1996 15572.1133 0.0600 24.1980 7.8017
MXNDArrayGetDType 1996 0.7260 0.0000 0.0010 0.0004
MXNDArrayGetShape 5050 2.3460 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2116 18.5810 0.0070 0.0280 0.0088
MXNDArrayFree 23734 243.3020 0.0000 0.9140 0.0103
MXNet C API Calls 106966 594.3230 487.3580 594.3230 53.4825
MXNet C API Concurrency 213931 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1876 0.8880 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9380 4.8350 0.0000 0.0230 0.0005
MXNDArraySetGradState 9380 4.7720 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1876 0.8060 0.0000 0.0010 0.0004
MXAutogradBackwardEx 938 518.0180 0.5230 0.9450 0.5523
MXNDArrayGetContext 14692 6.5030 0.0000 0.0190 0.0004
MXImperativeInvokeEx 30998 796.5020 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3752 9165.0781 0.7010 3.7500 2.4427
FullyConnected 6348 613.2870 0.0080 0.3430 0.0966
argmax 2116 41.9460 0.0050 0.0480 0.0198
_backward_pick 1876 29.7600 0.0130 0.0320 0.0159
_backward_mul_scalar 1876 8.1960 0.0030 0.0220 0.0044
DeleteVariable 79360 561.1570 0.0010 0.9010 0.0071
Flatten 2116 88.0680 0.0080 0.1120 0.0416
Activation 8464 392.8080 0.0030 1.9440 0.0464
SetValueOp 1876 12.1910 0.0050 0.0180 0.0065
pick 1876 14.7950 0.0060 0.0280 0.0079
Pooling 4232 5869.5342 0.0640 3.1720 1.3869
Convolution 4232 23991.0742 0.2260 13.2050 5.6690
CopyCPU2CPU 2116 11.9830 0.0030 0.0360 0.0057
log_softmax 1876 58.5580 0.0150 0.0550 0.0312
_backward_copy 1876 97.2110 0.0220 0.0760 0.0518
_backward_Convolution 3752 34692.8008 2.6240 12.5610 9.2465
_mul_scalar 1876 6.6880 0.0020 0.0230 0.0036
_backward_log_softmax 1876 32.1020 0.0090 0.0400 0.0171
mean 5868 112.0500 0.0080 0.0730 0.0191
_backward_mean 1876 20.5650 0.0080 0.1030 0.0110
_backward_FullyConnected 5628 1079.9220 0.0280 0.5940 0.1919
WaitForVar 3992 35.5320 0.0060 0.0280 0.0089
broadcast_equal 2116 10.5430 0.0030 0.0110 0.0050
sgd_update 18760 188.1790 0.0020 0.0790 0.0100
_backward_Activation 7504 554.0650 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 73086 55485.6250 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6354 1557.4399 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 938 3724.2859 0.3060 73.5870 3.9705
MXNDArrayCreateEx 1059 36.5990 0.0310 0.0640 0.0346
MXNDArrayWaitAll 939 20379.1680 5.6130 25.0610 21.7031
MXNDArraySyncCopyToCPU 1998 15586.3711 0.0600 24.1980 7.8010
MXNDArrayGetDType 1998 0.7270 0.0000 0.0010 0.0004
MXNDArrayGetShape 5055 2.3480 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2118 18.5980 0.0070 0.0280 0.0088
MXNDArrayFree 23757 243.5140 0.0000 0.9140 0.0103
MXNet C API Calls 107073 594.4300 487.3580 594.4300 53.5360
MXNet C API Concurrency 214145 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1878 0.8890 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9390 4.8410 0.0000 0.0230 0.0005
MXNDArraySetGradState 9390 4.7770 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1878 0.8070 0.0000 0.0010 0.0004
MXAutogradBackwardEx 939 518.5690 0.5230 0.9450 0.5523
MXNDArrayGetContext 14706 6.5080 0.0000 0.0190 0.0004
MXImperativeInvokeEx 31029 797.3040 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3756 9174.7607 0.7010 3.7500 2.4427
FullyConnected 6354 613.8160 0.0080 0.3430 0.0966
argmax 2118 41.9830 0.0050 0.0480 0.0198
_backward_pick 1878 29.7900 0.0130 0.0320 0.0159
_backward_mul_scalar 1878 8.2040 0.0030 0.0220 0.0044
DeleteVariable 79440 561.6650 0.0010 0.9010 0.0071
Flatten 2118 88.1440 0.0080 0.1120 0.0416
Activation 8472 393.1570 0.0030 1.9440 0.0464
SetValueOp 1878 12.2030 0.0050 0.0180 0.0065
pick 1878 14.8110 0.0060 0.0280 0.0079
Pooling 4236 5874.9160 0.0640 3.1720 1.3869
Convolution 4236 24013.1738 0.2260 13.2050 5.6688
CopyCPU2CPU 2118 11.9920 0.0030 0.0360 0.0057
log_softmax 1878 58.6180 0.0150 0.0550 0.0312
_backward_copy 1878 97.3140 0.0220 0.0760 0.0518
_backward_Convolution 3756 34729.6055 2.6240 12.5610 9.2464
_mul_scalar 1878 6.6950 0.0020 0.0230 0.0036
_backward_log_softmax 1878 32.1340 0.0090 0.0400 0.0171
mean 5874 112.1220 0.0080 0.0730 0.0191
_backward_mean 1878 20.5850 0.0080 0.1030 0.0110
_backward_FullyConnected 5634 1080.9830 0.0280 0.5940 0.1919
WaitForVar 3996 35.5680 0.0060 0.0280 0.0089
broadcast_equal 2118 10.5520 0.0030 0.0110 0.0050
sgd_update 18780 188.3270 0.0020 0.0790 0.0100
_backward_Activation 7512 554.6910 0.0070 0.3900 0.0738
Profile Statistics.
Note that counter items are counter values and not time units.
Device Storage
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
Memory: cpu/0 73160 55478.5859 55478.5859 66102.4609 5311.9380
Memory: cpu shared/ 6360 1055.0400 803.8400 2361.2800 778.7200
MXNET_C_API
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
MXAggregateProfileStatsPrint 939 3728.1819 0.3060 73.5870 3.9704
MXNDArrayCreateEx 1060 36.6320 0.0310 0.0640 0.0346
MXNDArrayWaitAll 940 20387.4570 5.6130 25.0610 21.6888
MXNDArraySyncCopyToCPU 2000 15590.3945 0.0600 24.1980 7.7952
MXNDArrayGetDType 2000 0.7280 0.0000 0.0010 0.0004
MXNDArrayGetShape 5060 2.3510 0.0000 0.0230 0.0005
MXNDArrayCreateFromSharedMem 2120 18.6140 0.0070 0.0280 0.0088
MXNDArrayFree 23780 243.7300 0.0000 0.9140 0.0102
MXNet C API Calls 107180 594.5370 487.3580 594.5370 53.5895
MXNet C API Concurrency 214359 0.0010 0.0000 0.0010 0.0005
MXAutogradSetIsRecording 1880 0.8900 0.0000 0.0040 0.0005
MXNDArrayGetGradState 9400 4.8450 0.0000 0.0230 0.0005
MXNDArraySetGradState 9400 4.7820 0.0000 0.0010 0.0005
MXAutogradSetIsTraining 1880 0.8070 0.0000 0.0010 0.0004
MXAutogradBackwardEx 940 519.0970 0.5230 0.9450 0.5522
MXNDArrayGetContext 14720 6.5140 0.0000 0.0190 0.0004
MXImperativeInvokeEx 31060 798.0780 0.0080 0.2420 0.0257
operator
=================
Name Total Count Time (ms) Min Time (ms) Max Time (ms) Avg Time (ms)
---- ----------- --------- ------------- ------------- -------------
_backward_Pooling 3760 9178.4971 0.6990 3.7500 2.4411
FullyConnected 6360 614.1250 0.0080 0.3430 0.0966
argmax 2120 42.0040 0.0050 0.0480 0.0198
_backward_pick 1880 29.8150 0.0120 0.0320 0.0159
_backward_mul_scalar 1880 8.2110 0.0030 0.0220 0.0044
DeleteVariable 79520 562.1540 0.0010 0.9010 0.0071
Flatten 2120 88.1830 0.0080 0.1120 0.0416
Activation 8480 393.3280 0.0030 1.9440 0.0464
SetValueOp 1880 12.2140 0.0050 0.0180 0.0065
pick 1880 14.8230 0.0050 0.0280 0.0079
Pooling 4240 5876.9399 0.0640 3.1720 1.3861
Convolution 4240 24021.5449 0.2260 13.2050 5.6655
CopyCPU2CPU 2120 12.0010 0.0030 0.0360 0.0057
log_softmax 1880 58.6470 0.0140 0.0550 0.0312
_backward_copy 1880 97.3600 0.0220 0.0760 0.0518
_backward_Convolution 3760 34743.5664 2.6240 12.5610 9.2403
_mul_scalar 1880 6.7020 0.0020 0.0230 0.0036
_backward_log_softmax 1880 32.1530 0.0090 0.0400 0.0171
mean 5880 112.1760 0.0080 0.0730 0.0191
_backward_mean 1880 20.6030 0.0080 0.1030 0.0110
_backward_FullyConnected 5640 1081.4910 0.0280 0.5940 0.1918
WaitForVar 4000 35.6050 0.0060 0.0280 0.0089
broadcast_equal 2120 10.5600 0.0030 0.0110 0.0050
sgd_update 18800 188.4750 0.0020 0.0790 0.0100
_backward_Activation 7520 554.9280 0.0070 0.3900 0.0738
Epoch 1: loss 0.171, train_acc 0.936, test acc 0.904, in 11.5 sec
help(mx.nd.waitall)
Help on function waitall in module mxnet.ndarray.ndarray:
waitall()
Wait for all async operations to finish in MXNet.
This function is used for benchmarking only.
My Podcast!
If you like topics such as this then please consider subscribing to my podcast. I talk to some of the stalwarts in tech and ask them what their favorite productivity hacks are:
Available on iTunes Podcast
Visit Void Star Podcast’s page on iTunes Podcast Portal. Please Click ‘Subscribe’, leave a comment.
