MINIST手写数字识别：分类应用入门

2022-01-12
Author 孙新斌
~31.25K words

1. MNIST手写数字识别问题

用神经元处理分类问题

MNIST手写数字识别问题

分类问题

MNIST手写数字识别数据集
MNIST 数据集来自美国国家标准与技术研究所, National Institute of Standards and Technology (NIST).
数据集由来自 250 个不同人手写的数字构成, 其中 50% 是高中学生, 50% 来自人口普查局 (the Census Bureau) 的工作人员
训练集 55000 验证集 5000 测试集 10000

数据集读取方法

import tensorflow as tf
import tensorflow.examples.tutorials.mnist.input_data as input_data
#10分类采用One Hot编码
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

了解数据集

print('训练集train数量：',mnist.train.num_examples,
     ',验证集validation数量：',mnist.validation.num_examples,
     ',测试集test数量：',mnist.test.num_examples)

print('train images shape:',mnist.train.images.shape,
     'labels shape:',mnist.train.labels.shape)

len(mnist.train.images[0])

mnist.train.images[0].shape

#Image数据再塑形reshape
mnist.train.images[0].reshape(28,28)

可视化image
plt.imshow()第二个参数是这个图像的模式参数，“binary”表示以灰度模式显示。
plt.imshow()函数中的图像数据参数支持一下数据形状：
•（M，N）：二维数值，代表图像大小为M行N列，值为每个像素点的取值。
•（M，N，3）：三维度数值，代表图像大小为M行N列（即图片的高和宽），每个像素点的取值具有RGB三个通道的值（float或uint8）。
• 参数cmap缺省值为none，将把图像数据映射为彩色图显示

import matplotlib.pyplot as plt

def plot_image(image):
    plt.imshow(image.reshape(28,28),cmap='binary')
    plt.show()

独热编码

一种稀疏向量，其中：一个元素设为 1，所有其他元素均设为 0
独热编码常用于表示拥有有限个可能值的字符串或标识符
例如：假设某个植物学数据集记录了 15000 个不同的物种，其中每个物种都用独一无二的字符串标识符来表示。在特征工程过程中，可能需要将这些字符串标识符编码为独热向量，向量的大小为 15000
使用独热编码的原因
1 将离散特征的取值扩展到了欧式空间，离散特征的某个取值就对应欧式空间的某个点
2 机器学习算法中，特征之间距离的计算或相似度的常用计算方法都是基于欧式空间的
3 将离散型特征使用one-hot编码，会让特征之间的距离计算更加合理

数据集划分

构建和训练机器学习模型是希望对新的数据做出良好预测
如何去保证训练的实效，可以应对以前未见过的数据呢？
一种方法是将数据集分成两个子集：
训练集 - 用于训练模型的子集
测试集 - 用于测试模型的子集
通常，在测试集上表现是否良好是衡量能否在新数据上表现良好的有用指标，前提是：
测试集足够大
不会反复使用相同的测试集来作假

拆分数据
将单个数据集拆分为一个训练集和一个测试集
确保测试集满足以下两个条件：
规模足够大，可产生具有统计意义的结果
能代表整个数据集，测试集的特征应该与训练集的特征相同

工作流程
work

问题：多次重复执行该流程可能导致模型不知不觉地拟合了特定测试集的特性

新的工作流程
new work
前向计算与结果分类
softmax

逻辑回归

许多问题的预测结果是一个在连续空间的数值，比如房价预测问题，可以用线性模型来描述
但也有很多场景需要输出的是概率估算值，例如：
• 根据邮件内容判断是垃圾邮件的可能性
• 根据医学影像判断肿瘤是恶性的可能性
• 手写数字分别是 0、1、2、3、4、5、6、7、8、9的可能性（概率）
这时需要将预测输出值控制在 [0，1]区间内
二元分类问题的目标是正确预测两个可能的标签中的一个
逻辑回归（Logistic Regression）可以用于处理这类问题
Sigmod函数
sigmod
逻辑回归中的损失函数

多元分类

Softmax 思想
逻辑回归可生成介于 0 和 1.0 之间的小数。
例如，某电子邮件分类器的逻辑回归输出值为 0.8，表明电子邮件是垃圾邮件的概率为80%，不是垃圾邮件的概率为 20%。很明显，一封电子邮件是垃圾邮件或非垃圾邮件的概率之和为 1.0。
Softmax将这一想法延伸到多类别领域。
在多类别问题中，Softmax会为每个类别分配一个用小数表示的概率。这些用小数表示的概率相加之和必须是 1.0。
softmax方程式
交叉熵损失函数
交叉熵是一个信息论中的概念，它原来是用来估算平均编码长度的。给定两个概率分布p和q，通过q来表示p的交叉熵为

交叉熵刻画的是两个概率分布之间的距离，p代表正确答案，q代表的是预测值，交叉熵越小，两个概率的分布约接近
完整程序

import tensorflow as tf
import tensorflow.examples.tutorials.mnist.input_data as input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

import matplotlib.pyplot as plt

def plot_image(image):
    plt.imshow(image.reshape(28,28),cmap='binary')
    plt.show()

#定义带输入数据的占位符
#784个像素点
x = tf.placeholder(tf.float32,[None, 784], name="X")
#10个类别
y = tf.placeholder(tf.float32,[None,10], name="Y")

#定义模型变量
#正态分布随机数初始化权重w，常数0初始化偏置b
W = tf.Variable(tf.random_normal([784, 10]), name="W")
b = tf.Variable(tf.zeros([10]),name="b")

#定义前向计算
forward = tf.matmul(x,W) + b

#结果分类（采用softmax分类）
pred = tf.nn.softmax(forward)

train_epochs = 500
batch_size = 100
total_batch = int(mnist.train.num_examples/batch_size)
display_step = 1
learning_rate = 0.01

#交叉熵
loss_function = tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred),
                            reduction_indices=1))

#选择优化器                                   
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss_function)

#检查预测类别与实际类别的匹配情况
correct_prediction = tf.equal(tf.argmax(pred,1),tf.argmax(y,1))

#准确率
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))

#声明会话，初始化变量
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)

#开始训练
for epoch in range (train_epochs):
    for batch in range(total_batch):
        xs,ys = mnist.train.next_batch(batch_size)#读取批次数据
        sess.run(optimizer,feed_dict={x:xs,y:ys})#执行训练
    #验证数据计算误差与准确率
    loss,acc = sess.run([loss_function,accuracy],
                       feed_dict={x:mnist.validation.images,y:mnist.validation.labels})
    #打印详细信息
    if(epoch+1) % display_step == 0:
        print("Train Epoch:",'%02d' % (epoch+1),"Loss=","{:.9f}".format(loss),\
             "Accuracy=","{:.4f}".format(acc))
        
print("Train Finshied!")

#评估模型
accu_test = sess.run(accuracy,
                    feed_dict={x:mnist.test.images,y:mnist.test.labels})
print("Test Accurary:",accu_test)

#应用模型
prediction_result = sess.run(tf.argmax(pred,1),
                            feed_dict = {x:mnist.test.images})

#查看预测结果
prediction_result[0:10]

#定义可视化函数
import matplotlib.pyplot as plot
import numpy as np
def plot_images_labels_prediction(images,
                                 labels,
                                 prediction,
                                 index,
                                 num=10):
    fig = plt.gcf()
    fig.set_size_inches(10, 12)
    if num > 25 :
        num = 25
    for i in range(0,num):
        ax = plt.subplot(5,5,i+1)
        
        ax.imshow(np.reshape(images[index],(28,28)),
                 cmap='binary')
        
        title = "label=" + str(np.argmax(labels[index]))
        if len(prediction)>0:
            title += ",predict=" + str(prediction[index])
            
        ax.set_title(title,fontsize=10)
        ax.set_xticks([]);
        ax.set_yticks([])
        index += 1
    plt.show()

#可视化预测结果
plot_images_labels_prediction(mnist.test.images,
                             mnist.test.labels,
                             prediction_result,10,15)

训练结果如下显示：（多数据警告）

训练结果

Train Epoch: 01 Loss= 6.405103683 Accuracy= 0.2280
Train Epoch: 02 Loss= 3.764361620 Accuracy= 0.4148
Train Epoch: 03 Loss= 2.728085279 Accuracy= 0.5380
Train Epoch: 04 Loss= 2.206749916 Accuracy= 0.6040
Train Epoch: 05 Loss= 1.891477942 Accuracy= 0.6544
Train Epoch: 06 Loss= 1.679210782 Accuracy= 0.6842
Train Epoch: 07 Loss= 1.525882602 Accuracy= 0.7104
Train Epoch: 08 Loss= 1.408041358 Accuracy= 0.7276
Train Epoch: 09 Loss= 1.315110087 Accuracy= 0.7408
Train Epoch: 10 Loss= 1.239514709 Accuracy= 0.7536
Train Epoch: 11 Loss= 1.177493334 Accuracy= 0.7648
Train Epoch: 12 Loss= 1.123586535 Accuracy= 0.7726
Train Epoch: 13 Loss= 1.077311754 Accuracy= 0.7818
Train Epoch: 14 Loss= 1.036843181 Accuracy= 0.7892
Train Epoch: 15 Loss= 1.002879262 Accuracy= 0.7950
Train Epoch: 16 Loss= 0.971034110 Accuracy= 0.8008
Train Epoch: 17 Loss= 0.943259358 Accuracy= 0.8056
Train Epoch: 18 Loss= 0.917949319 Accuracy= 0.8094
Train Epoch: 19 Loss= 0.894973814 Accuracy= 0.8132
Train Epoch: 20 Loss= 0.874269903 Accuracy= 0.8188
Train Epoch: 21 Loss= 0.854999602 Accuracy= 0.8222
Train Epoch: 22 Loss= 0.837613404 Accuracy= 0.8258
Train Epoch: 23 Loss= 0.821559370 Accuracy= 0.8272
Train Epoch: 24 Loss= 0.807139874 Accuracy= 0.8294
Train Epoch: 25 Loss= 0.792462468 Accuracy= 0.8322
Train Epoch: 26 Loss= 0.779907167 Accuracy= 0.8336
Train Epoch: 27 Loss= 0.767695904 Accuracy= 0.8372
Train Epoch: 28 Loss= 0.756269753 Accuracy= 0.8376
Train Epoch: 29 Loss= 0.745229483 Accuracy= 0.8398
Train Epoch: 30 Loss= 0.735225081 Accuracy= 0.8424
Train Epoch: 31 Loss= 0.725513577 Accuracy= 0.8442
Train Epoch: 32 Loss= 0.716079116 Accuracy= 0.8466
Train Epoch: 33 Loss= 0.708266139 Accuracy= 0.8472
Train Epoch: 34 Loss= 0.700395644 Accuracy= 0.8482
Train Epoch: 35 Loss= 0.692030370 Accuracy= 0.8500
Train Epoch: 36 Loss= 0.684325218 Accuracy= 0.8524
Train Epoch: 37 Loss= 0.677517653 Accuracy= 0.8520
Train Epoch: 38 Loss= 0.671082914 Accuracy= 0.8536
Train Epoch: 39 Loss= 0.664354265 Accuracy= 0.8546
Train Epoch: 40 Loss= 0.658523440 Accuracy= 0.8554
Train Epoch: 41 Loss= 0.652254462 Accuracy= 0.8562
Train Epoch: 42 Loss= 0.646569550 Accuracy= 0.8564
Train Epoch: 43 Loss= 0.640967071 Accuracy= 0.8562
Train Epoch: 44 Loss= 0.635610282 Accuracy= 0.8568
Train Epoch: 45 Loss= 0.630128682 Accuracy= 0.8588
Train Epoch: 46 Loss= 0.625474632 Accuracy= 0.8600
Train Epoch: 47 Loss= 0.620498955 Accuracy= 0.8608
Train Epoch: 48 Loss= 0.616365254 Accuracy= 0.8608
Train Epoch: 49 Loss= 0.612295687 Accuracy= 0.8622
Train Epoch: 50 Loss= 0.607778609 Accuracy= 0.8626
Train Epoch: 51 Loss= 0.603170395 Accuracy= 0.8644
Train Epoch: 52 Loss= 0.599933922 Accuracy= 0.8648
Train Epoch: 53 Loss= 0.595565438 Accuracy= 0.8670
Train Epoch: 54 Loss= 0.591758490 Accuracy= 0.8680
Train Epoch: 55 Loss= 0.588303804 Accuracy= 0.8694
Train Epoch: 56 Loss= 0.585322618 Accuracy= 0.8696
Train Epoch: 57 Loss= 0.581881523 Accuracy= 0.8698
Train Epoch: 58 Loss= 0.577679873 Accuracy= 0.8706
Train Epoch: 59 Loss= 0.574802935 Accuracy= 0.8716
Train Epoch: 60 Loss= 0.572479665 Accuracy= 0.8724
Train Epoch: 61 Loss= 0.568615794 Accuracy= 0.8730
Train Epoch: 62 Loss= 0.566718578 Accuracy= 0.8728
Train Epoch: 63 Loss= 0.563213229 Accuracy= 0.8730
Train Epoch: 64 Loss= 0.559740126 Accuracy= 0.8732
Train Epoch: 65 Loss= 0.556897581 Accuracy= 0.8732
Train Epoch: 66 Loss= 0.554247439 Accuracy= 0.8750
Train Epoch: 67 Loss= 0.551319838 Accuracy= 0.8754
Train Epoch: 68 Loss= 0.548921108 Accuracy= 0.8742
Train Epoch: 69 Loss= 0.546500266 Accuracy= 0.8752
Train Epoch: 70 Loss= 0.543983519 Accuracy= 0.8758
Train Epoch: 71 Loss= 0.542061985 Accuracy= 0.8758
Train Epoch: 72 Loss= 0.539074361 Accuracy= 0.8768
Train Epoch: 73 Loss= 0.536839545 Accuracy= 0.8766
Train Epoch: 74 Loss= 0.534821987 Accuracy= 0.8764
Train Epoch: 75 Loss= 0.533168137 Accuracy= 0.8774
Train Epoch: 76 Loss= 0.530308068 Accuracy= 0.8784
Train Epoch: 77 Loss= 0.528434753 Accuracy= 0.8778
Train Epoch: 78 Loss= 0.526309192 Accuracy= 0.8788
Train Epoch: 79 Loss= 0.524245977 Accuracy= 0.8782
Train Epoch: 80 Loss= 0.522068501 Accuracy= 0.8792
Train Epoch: 81 Loss= 0.520250797 Accuracy= 0.8794
Train Epoch: 82 Loss= 0.517979860 Accuracy= 0.8794
Train Epoch: 83 Loss= 0.516227961 Accuracy= 0.8806
Train Epoch: 84 Loss= 0.514664233 Accuracy= 0.8802
Train Epoch: 85 Loss= 0.512204289 Accuracy= 0.8802
Train Epoch: 86 Loss= 0.510789692 Accuracy= 0.8814
Train Epoch: 87 Loss= 0.509589970 Accuracy= 0.8810
Train Epoch: 88 Loss= 0.507072687 Accuracy= 0.8816
Train Epoch: 89 Loss= 0.505876541 Accuracy= 0.8824
Train Epoch: 90 Loss= 0.504153013 Accuracy= 0.8822
Train Epoch: 91 Loss= 0.502235591 Accuracy= 0.8832
Train Epoch: 92 Loss= 0.501242876 Accuracy= 0.8828
Train Epoch: 93 Loss= 0.498941988 Accuracy= 0.8836
Train Epoch: 94 Loss= 0.497406751 Accuracy= 0.8832
Train Epoch: 95 Loss= 0.496222556 Accuracy= 0.8838
Train Epoch: 96 Loss= 0.494214565 Accuracy= 0.8840
Train Epoch: 97 Loss= 0.493356436 Accuracy= 0.8832
Train Epoch: 98 Loss= 0.491570175 Accuracy= 0.8846
Train Epoch: 99 Loss= 0.489944965 Accuracy= 0.8840
Train Epoch: 100 Loss= 0.488946378 Accuracy= 0.8842
Train Epoch: 101 Loss= 0.487533092 Accuracy= 0.8844
Train Epoch: 102 Loss= 0.485964954 Accuracy= 0.8852
Train Epoch: 103 Loss= 0.484404892 Accuracy= 0.8852
Train Epoch: 104 Loss= 0.483356357 Accuracy= 0.8846
Train Epoch: 105 Loss= 0.482004821 Accuracy= 0.8860
Train Epoch: 106 Loss= 0.480790615 Accuracy= 0.8854
Train Epoch: 107 Loss= 0.479260147 Accuracy= 0.8858
Train Epoch: 108 Loss= 0.478474498 Accuracy= 0.8860
Train Epoch: 109 Loss= 0.476907611 Accuracy= 0.8862
Train Epoch: 110 Loss= 0.475418597 Accuracy= 0.8868
Train Epoch: 111 Loss= 0.474131107 Accuracy= 0.8870
Train Epoch: 112 Loss= 0.473437607 Accuracy= 0.8874
Train Epoch: 113 Loss= 0.471899986 Accuracy= 0.8874
Train Epoch: 114 Loss= 0.470724523 Accuracy= 0.8882
Train Epoch: 115 Loss= 0.469866693 Accuracy= 0.8880
Train Epoch: 116 Loss= 0.468689829 Accuracy= 0.8872
Train Epoch: 117 Loss= 0.468155473 Accuracy= 0.8878
Train Epoch: 118 Loss= 0.466815442 Accuracy= 0.8886
Train Epoch: 119 Loss= 0.465229392 Accuracy= 0.8882
Train Epoch: 120 Loss= 0.464131504 Accuracy= 0.8890
Train Epoch: 121 Loss= 0.462855667 Accuracy= 0.8886
Train Epoch: 122 Loss= 0.462298840 Accuracy= 0.8880
Train Epoch: 123 Loss= 0.461265326 Accuracy= 0.8894
Train Epoch: 124 Loss= 0.460113525 Accuracy= 0.8894
Train Epoch: 125 Loss= 0.459472865 Accuracy= 0.8892
Train Epoch: 126 Loss= 0.458169192 Accuracy= 0.8894
Train Epoch: 127 Loss= 0.456787854 Accuracy= 0.8898
Train Epoch: 128 Loss= 0.456247598 Accuracy= 0.8892
Train Epoch: 129 Loss= 0.455326319 Accuracy= 0.8898
Train Epoch: 130 Loss= 0.453999221 Accuracy= 0.8904
Train Epoch: 131 Loss= 0.453291893 Accuracy= 0.8900
Train Epoch: 132 Loss= 0.452165246 Accuracy= 0.8900
Train Epoch: 133 Loss= 0.451443315 Accuracy= 0.8906
Train Epoch: 134 Loss= 0.450718075 Accuracy= 0.8898
Train Epoch: 135 Loss= 0.449742883 Accuracy= 0.8906
Train Epoch: 136 Loss= 0.448562503 Accuracy= 0.8908
Train Epoch: 137 Loss= 0.447787493 Accuracy= 0.8912
Train Epoch: 138 Loss= 0.447070807 Accuracy= 0.8920
Train Epoch: 139 Loss= 0.446267515 Accuracy= 0.8922
Train Epoch: 140 Loss= 0.445026428 Accuracy= 0.8930
Train Epoch: 141 Loss= 0.444738477 Accuracy= 0.8918
Train Epoch: 142 Loss= 0.443849623 Accuracy= 0.8916
Train Epoch: 143 Loss= 0.442687124 Accuracy= 0.8924
Train Epoch: 144 Loss= 0.442112952 Accuracy= 0.8924
Train Epoch: 145 Loss= 0.441391319 Accuracy= 0.8924
Train Epoch: 146 Loss= 0.440162748 Accuracy= 0.8926
Train Epoch: 147 Loss= 0.439293742 Accuracy= 0.8934
Train Epoch: 148 Loss= 0.438909620 Accuracy= 0.8930
Train Epoch: 149 Loss= 0.437819093 Accuracy= 0.8934
Train Epoch: 150 Loss= 0.437358975 Accuracy= 0.8934
Train Epoch: 151 Loss= 0.436241060 Accuracy= 0.8936
Train Epoch: 152 Loss= 0.436019480 Accuracy= 0.8940
Train Epoch: 153 Loss= 0.435418934 Accuracy= 0.8940
Train Epoch: 154 Loss= 0.434051841 Accuracy= 0.8948
Train Epoch: 155 Loss= 0.433334082 Accuracy= 0.8946
Train Epoch: 156 Loss= 0.432984859 Accuracy= 0.8944
Train Epoch: 157 Loss= 0.432401121 Accuracy= 0.8946
Train Epoch: 158 Loss= 0.431591660 Accuracy= 0.8944
Train Epoch: 159 Loss= 0.430783927 Accuracy= 0.8950
Train Epoch: 160 Loss= 0.430064738 Accuracy= 0.8954
Train Epoch: 161 Loss= 0.428908020 Accuracy= 0.8958
Train Epoch: 162 Loss= 0.428390443 Accuracy= 0.8956
Train Epoch: 163 Loss= 0.428132027 Accuracy= 0.8962
Train Epoch: 164 Loss= 0.427713633 Accuracy= 0.8958
Train Epoch: 165 Loss= 0.426613778 Accuracy= 0.8960
Train Epoch: 166 Loss= 0.425659865 Accuracy= 0.8964
Train Epoch: 167 Loss= 0.425214738 Accuracy= 0.8964
Train Epoch: 168 Loss= 0.424902737 Accuracy= 0.8960
Train Epoch: 169 Loss= 0.424074769 Accuracy= 0.8966
Train Epoch: 170 Loss= 0.423162699 Accuracy= 0.8974
Train Epoch: 171 Loss= 0.422749072 Accuracy= 0.8974
Train Epoch: 172 Loss= 0.422401696 Accuracy= 0.8964
Train Epoch: 173 Loss= 0.421341300 Accuracy= 0.8976
Train Epoch: 174 Loss= 0.420944721 Accuracy= 0.8964
Train Epoch: 175 Loss= 0.419966906 Accuracy= 0.8976
Train Epoch: 176 Loss= 0.419914395 Accuracy= 0.8968
Train Epoch: 177 Loss= 0.418810934 Accuracy= 0.8970
Train Epoch: 178 Loss= 0.418342829 Accuracy= 0.8968
Train Epoch: 179 Loss= 0.417744726 Accuracy= 0.8970
Train Epoch: 180 Loss= 0.417116791 Accuracy= 0.8978
Train Epoch: 181 Loss= 0.416599661 Accuracy= 0.8976
Train Epoch: 182 Loss= 0.415944725 Accuracy= 0.8982
Train Epoch: 183 Loss= 0.415637791 Accuracy= 0.8978
Train Epoch: 184 Loss= 0.414836079 Accuracy= 0.8980
Train Epoch: 185 Loss= 0.414405614 Accuracy= 0.8978
Train Epoch: 186 Loss= 0.413655072 Accuracy= 0.8982
Train Epoch: 187 Loss= 0.413016111 Accuracy= 0.8982
Train Epoch: 188 Loss= 0.412935942 Accuracy= 0.8988
Train Epoch: 189 Loss= 0.412001669 Accuracy= 0.8984
Train Epoch: 190 Loss= 0.411753237 Accuracy= 0.8992
Train Epoch: 191 Loss= 0.411443263 Accuracy= 0.8990
Train Epoch: 192 Loss= 0.410422057 Accuracy= 0.8984
Train Epoch: 193 Loss= 0.410251558 Accuracy= 0.8996
Train Epoch: 194 Loss= 0.409084409 Accuracy= 0.8988
Train Epoch: 195 Loss= 0.408952326 Accuracy= 0.8998
Train Epoch: 196 Loss= 0.408304602 Accuracy= 0.8998
Train Epoch: 197 Loss= 0.407955080 Accuracy= 0.8992
Train Epoch: 198 Loss= 0.407745570 Accuracy= 0.8992
Train Epoch: 199 Loss= 0.406976283 Accuracy= 0.8994
Train Epoch: 200 Loss= 0.406263798 Accuracy= 0.8996
Train Epoch: 201 Loss= 0.406233251 Accuracy= 0.8994
Train Epoch: 202 Loss= 0.405623794 Accuracy= 0.9000
Train Epoch: 203 Loss= 0.404749840 Accuracy= 0.8998
Train Epoch: 204 Loss= 0.404725671 Accuracy= 0.9006
Train Epoch: 205 Loss= 0.403718770 Accuracy= 0.9002
Train Epoch: 206 Loss= 0.403355181 Accuracy= 0.9006
Train Epoch: 207 Loss= 0.402798116 Accuracy= 0.9008
Train Epoch: 208 Loss= 0.402724475 Accuracy= 0.9002
Train Epoch: 209 Loss= 0.402527630 Accuracy= 0.9006
Train Epoch: 210 Loss= 0.401481658 Accuracy= 0.9006
Train Epoch: 211 Loss= 0.401139230 Accuracy= 0.9006
Train Epoch: 212 Loss= 0.400432110 Accuracy= 0.9004
Train Epoch: 213 Loss= 0.400378883 Accuracy= 0.9004
Train Epoch: 214 Loss= 0.399567872 Accuracy= 0.9002
Train Epoch: 215 Loss= 0.399531484 Accuracy= 0.9006
Train Epoch: 216 Loss= 0.399013489 Accuracy= 0.9008
Train Epoch: 217 Loss= 0.397973686 Accuracy= 0.9012
Train Epoch: 218 Loss= 0.397794008 Accuracy= 0.9008
Train Epoch: 219 Loss= 0.397283971 Accuracy= 0.9012
Train Epoch: 220 Loss= 0.397037268 Accuracy= 0.9014
Train Epoch: 221 Loss= 0.396341175 Accuracy= 0.9020
Train Epoch: 222 Loss= 0.396116793 Accuracy= 0.9002
Train Epoch: 223 Loss= 0.395749956 Accuracy= 0.9014
Train Epoch: 224 Loss= 0.395612061 Accuracy= 0.9008
Train Epoch: 225 Loss= 0.394687176 Accuracy= 0.9012
Train Epoch: 226 Loss= 0.394498616 Accuracy= 0.9016
Train Epoch: 227 Loss= 0.394306749 Accuracy= 0.9012
Train Epoch: 228 Loss= 0.393707484 Accuracy= 0.9008
Train Epoch: 229 Loss= 0.393643707 Accuracy= 0.9008
Train Epoch: 230 Loss= 0.393045634 Accuracy= 0.9008
Train Epoch: 231 Loss= 0.392508119 Accuracy= 0.9022
Train Epoch: 232 Loss= 0.391976804 Accuracy= 0.9018
Train Epoch: 233 Loss= 0.391808689 Accuracy= 0.9006
Train Epoch: 234 Loss= 0.391237199 Accuracy= 0.9014
Train Epoch: 235 Loss= 0.390866280 Accuracy= 0.9006
Train Epoch: 236 Loss= 0.390418172 Accuracy= 0.9010
Train Epoch: 237 Loss= 0.389898777 Accuracy= 0.9024
Train Epoch: 238 Loss= 0.389768124 Accuracy= 0.9020
Train Epoch: 239 Loss= 0.389144570 Accuracy= 0.9020
Train Epoch: 240 Loss= 0.389046282 Accuracy= 0.9030
Train Epoch: 241 Loss= 0.388572276 Accuracy= 0.9024
Train Epoch: 242 Loss= 0.388103604 Accuracy= 0.9018
Train Epoch: 243 Loss= 0.387917459 Accuracy= 0.9026
Train Epoch: 244 Loss= 0.387428015 Accuracy= 0.9024
Train Epoch: 245 Loss= 0.387170792 Accuracy= 0.9032
Train Epoch: 246 Loss= 0.386788905 Accuracy= 0.9014
Train Epoch: 247 Loss= 0.386517614 Accuracy= 0.9030
Train Epoch: 248 Loss= 0.386347741 Accuracy= 0.9020
Train Epoch: 249 Loss= 0.385344386 Accuracy= 0.9036
Train Epoch: 250 Loss= 0.384959489 Accuracy= 0.9026
Train Epoch: 251 Loss= 0.384517908 Accuracy= 0.9028
Train Epoch: 252 Loss= 0.384441257 Accuracy= 0.9030
Train Epoch: 253 Loss= 0.383821398 Accuracy= 0.9028
Train Epoch: 254 Loss= 0.383479148 Accuracy= 0.9022
Train Epoch: 255 Loss= 0.383624524 Accuracy= 0.9032
Train Epoch: 256 Loss= 0.383323073 Accuracy= 0.9022
Train Epoch: 257 Loss= 0.382732749 Accuracy= 0.9028
Train Epoch: 258 Loss= 0.382320166 Accuracy= 0.9030
Train Epoch: 259 Loss= 0.381811768 Accuracy= 0.9028
Train Epoch: 260 Loss= 0.381984144 Accuracy= 0.9026
Train Epoch: 261 Loss= 0.381227553 Accuracy= 0.9026
Train Epoch: 262 Loss= 0.380849212 Accuracy= 0.9022
Train Epoch: 263 Loss= 0.380524695 Accuracy= 0.9040
Train Epoch: 264 Loss= 0.380101144 Accuracy= 0.9028
Train Epoch: 265 Loss= 0.380186915 Accuracy= 0.9024
Train Epoch: 266 Loss= 0.379487067 Accuracy= 0.9030
Train Epoch: 267 Loss= 0.379631072 Accuracy= 0.9034
Train Epoch: 268 Loss= 0.378731072 Accuracy= 0.9034
Train Epoch: 269 Loss= 0.378665388 Accuracy= 0.9030
Train Epoch: 270 Loss= 0.378257990 Accuracy= 0.9032
Train Epoch: 271 Loss= 0.377845258 Accuracy= 0.9038
Train Epoch: 272 Loss= 0.377908498 Accuracy= 0.9034
Train Epoch: 273 Loss= 0.377575696 Accuracy= 0.9038
Train Epoch: 274 Loss= 0.377002954 Accuracy= 0.9034
Train Epoch: 275 Loss= 0.376586318 Accuracy= 0.9034
Train Epoch: 276 Loss= 0.376820147 Accuracy= 0.9040
Train Epoch: 277 Loss= 0.376379251 Accuracy= 0.9036
Train Epoch: 278 Loss= 0.375819713 Accuracy= 0.9036
Train Epoch: 279 Loss= 0.375545800 Accuracy= 0.9046
Train Epoch: 280 Loss= 0.375153273 Accuracy= 0.9044
Train Epoch: 281 Loss= 0.375212997 Accuracy= 0.9034
Train Epoch: 282 Loss= 0.374674737 Accuracy= 0.9042
Train Epoch: 283 Loss= 0.374336630 Accuracy= 0.9046
Train Epoch: 284 Loss= 0.373770863 Accuracy= 0.9042
Train Epoch: 285 Loss= 0.373896927 Accuracy= 0.9040
Train Epoch: 286 Loss= 0.373323888 Accuracy= 0.9046
Train Epoch: 287 Loss= 0.373260558 Accuracy= 0.9046
Train Epoch: 288 Loss= 0.372779518 Accuracy= 0.9044
Train Epoch: 289 Loss= 0.372727990 Accuracy= 0.9046
Train Epoch: 290 Loss= 0.372335255 Accuracy= 0.9046
Train Epoch: 291 Loss= 0.372065455 Accuracy= 0.9048
Train Epoch: 292 Loss= 0.371591061 Accuracy= 0.9046
Train Epoch: 293 Loss= 0.371365815 Accuracy= 0.9046
Train Epoch: 294 Loss= 0.371317178 Accuracy= 0.9048
Train Epoch: 295 Loss= 0.370640844 Accuracy= 0.9052
Train Epoch: 296 Loss= 0.370910048 Accuracy= 0.9044
Train Epoch: 297 Loss= 0.370179355 Accuracy= 0.9050
Train Epoch: 298 Loss= 0.370271593 Accuracy= 0.9054
Train Epoch: 299 Loss= 0.369908154 Accuracy= 0.9054
Train Epoch: 300 Loss= 0.369609326 Accuracy= 0.9054
Train Epoch: 301 Loss= 0.369176894 Accuracy= 0.9054
Train Epoch: 302 Loss= 0.369029045 Accuracy= 0.9056
Train Epoch: 303 Loss= 0.368548244 Accuracy= 0.9058
Train Epoch: 304 Loss= 0.368454874 Accuracy= 0.9058
Train Epoch: 305 Loss= 0.368041515 Accuracy= 0.9056
Train Epoch: 306 Loss= 0.367680520 Accuracy= 0.9058
Train Epoch: 307 Loss= 0.367845863 Accuracy= 0.9056
Train Epoch: 308 Loss= 0.367201656 Accuracy= 0.9060
Train Epoch: 309 Loss= 0.366869599 Accuracy= 0.9062
Train Epoch: 310 Loss= 0.366847694 Accuracy= 0.9060
Train Epoch: 311 Loss= 0.366336137 Accuracy= 0.9058
Train Epoch: 312 Loss= 0.366104513 Accuracy= 0.9060
Train Epoch: 313 Loss= 0.366278410 Accuracy= 0.9058
Train Epoch: 314 Loss= 0.365434617 Accuracy= 0.9064
Train Epoch: 315 Loss= 0.365378529 Accuracy= 0.9064
Train Epoch: 316 Loss= 0.365533412 Accuracy= 0.9058
Train Epoch: 317 Loss= 0.365197808 Accuracy= 0.9068
Train Epoch: 318 Loss= 0.365037024 Accuracy= 0.9068
Train Epoch: 319 Loss= 0.364381582 Accuracy= 0.9064
Train Epoch: 320 Loss= 0.364105165 Accuracy= 0.9072
Train Epoch: 321 Loss= 0.364401549 Accuracy= 0.9066
Train Epoch: 322 Loss= 0.363872409 Accuracy= 0.9066
Train Epoch: 323 Loss= 0.363355607 Accuracy= 0.9072
Train Epoch: 324 Loss= 0.363645494 Accuracy= 0.9064
Train Epoch: 325 Loss= 0.362869859 Accuracy= 0.9070
Train Epoch: 326 Loss= 0.362704009 Accuracy= 0.9066
Train Epoch: 327 Loss= 0.362510622 Accuracy= 0.9072
Train Epoch: 328 Loss= 0.362664402 Accuracy= 0.9064
Train Epoch: 329 Loss= 0.362214684 Accuracy= 0.9070
Train Epoch: 330 Loss= 0.361763179 Accuracy= 0.9068
Train Epoch: 331 Loss= 0.361724973 Accuracy= 0.9072
Train Epoch: 332 Loss= 0.361439168 Accuracy= 0.9072
Train Epoch: 333 Loss= 0.361238927 Accuracy= 0.9072
Train Epoch: 334 Loss= 0.360923618 Accuracy= 0.9070
Train Epoch: 335 Loss= 0.360640526 Accuracy= 0.9070
Train Epoch: 336 Loss= 0.360468805 Accuracy= 0.9074
Train Epoch: 337 Loss= 0.360336691 Accuracy= 0.9070
Train Epoch: 338 Loss= 0.360181004 Accuracy= 0.9066
Train Epoch: 339 Loss= 0.359589636 Accuracy= 0.9078
Train Epoch: 340 Loss= 0.359605044 Accuracy= 0.9074
Train Epoch: 341 Loss= 0.359191000 Accuracy= 0.9068
Train Epoch: 342 Loss= 0.359398872 Accuracy= 0.9072
Train Epoch: 343 Loss= 0.358821988 Accuracy= 0.9072
Train Epoch: 344 Loss= 0.358554870 Accuracy= 0.9072
Train Epoch: 345 Loss= 0.358417094 Accuracy= 0.9074
Train Epoch: 346 Loss= 0.358287454 Accuracy= 0.9082
Train Epoch: 347 Loss= 0.358403027 Accuracy= 0.9074
Train Epoch: 348 Loss= 0.357745498 Accuracy= 0.9076
Train Epoch: 349 Loss= 0.357700974 Accuracy= 0.9074
Train Epoch: 350 Loss= 0.357280284 Accuracy= 0.9074
Train Epoch: 351 Loss= 0.357116640 Accuracy= 0.9072
Train Epoch: 352 Loss= 0.356674671 Accuracy= 0.9076
Train Epoch: 353 Loss= 0.356743395 Accuracy= 0.9080
Train Epoch: 354 Loss= 0.356310487 Accuracy= 0.9080
Train Epoch: 355 Loss= 0.356106997 Accuracy= 0.9080
Train Epoch: 356 Loss= 0.356351852 Accuracy= 0.9076
Train Epoch: 357 Loss= 0.356079578 Accuracy= 0.9078
Train Epoch: 358 Loss= 0.355536312 Accuracy= 0.9080
Train Epoch: 359 Loss= 0.355534166 Accuracy= 0.9080
Train Epoch: 360 Loss= 0.355263025 Accuracy= 0.9070
Train Epoch: 361 Loss= 0.355054259 Accuracy= 0.9086
Train Epoch: 362 Loss= 0.354682177 Accuracy= 0.9086
Train Epoch: 363 Loss= 0.354584038 Accuracy= 0.9084
Train Epoch: 364 Loss= 0.354257971 Accuracy= 0.9076
Train Epoch: 365 Loss= 0.354202539 Accuracy= 0.9084
Train Epoch: 366 Loss= 0.354148418 Accuracy= 0.9080
Train Epoch: 367 Loss= 0.353945524 Accuracy= 0.9072
Train Epoch: 368 Loss= 0.353680283 Accuracy= 0.9084
Train Epoch: 369 Loss= 0.353619725 Accuracy= 0.9088
Train Epoch: 370 Loss= 0.353221059 Accuracy= 0.9086
Train Epoch: 371 Loss= 0.353100061 Accuracy= 0.9084
Train Epoch: 372 Loss= 0.352835447 Accuracy= 0.9090
Train Epoch: 373 Loss= 0.352596819 Accuracy= 0.9084
Train Epoch: 374 Loss= 0.352595180 Accuracy= 0.9080
Train Epoch: 375 Loss= 0.352259755 Accuracy= 0.9090
Train Epoch: 376 Loss= 0.352301449 Accuracy= 0.9086
Train Epoch: 377 Loss= 0.352147013 Accuracy= 0.9088
Train Epoch: 378 Loss= 0.351966113 Accuracy= 0.9088
Train Epoch: 379 Loss= 0.351528525 Accuracy= 0.9088
Train Epoch: 380 Loss= 0.351112694 Accuracy= 0.9088
Train Epoch: 381 Loss= 0.350878716 Accuracy= 0.9088
Train Epoch: 382 Loss= 0.351027966 Accuracy= 0.9094
Train Epoch: 383 Loss= 0.350885630 Accuracy= 0.9092
Train Epoch: 384 Loss= 0.350694865 Accuracy= 0.9086
Train Epoch: 385 Loss= 0.350394100 Accuracy= 0.9084
Train Epoch: 386 Loss= 0.349927545 Accuracy= 0.9090
Train Epoch: 387 Loss= 0.350075901 Accuracy= 0.9092
Train Epoch: 388 Loss= 0.349799097 Accuracy= 0.9086
Train Epoch: 389 Loss= 0.349439144 Accuracy= 0.9098
Train Epoch: 390 Loss= 0.349565566 Accuracy= 0.9100
Train Epoch: 391 Loss= 0.349208802 Accuracy= 0.9102
Train Epoch: 392 Loss= 0.349225044 Accuracy= 0.9100
Train Epoch: 393 Loss= 0.349022955 Accuracy= 0.9106
Train Epoch: 394 Loss= 0.348788708 Accuracy= 0.9090
Train Epoch: 395 Loss= 0.348476678 Accuracy= 0.9092
Train Epoch: 396 Loss= 0.348356456 Accuracy= 0.9096
Train Epoch: 397 Loss= 0.348270595 Accuracy= 0.9094
Train Epoch: 398 Loss= 0.348110557 Accuracy= 0.9100
Train Epoch: 399 Loss= 0.347774327 Accuracy= 0.9114
Train Epoch: 400 Loss= 0.347881585 Accuracy= 0.9108
Train Epoch: 401 Loss= 0.347597957 Accuracy= 0.9106
Train Epoch: 402 Loss= 0.347694665 Accuracy= 0.9110
Train Epoch: 403 Loss= 0.347311169 Accuracy= 0.9116
Train Epoch: 404 Loss= 0.347082287 Accuracy= 0.9106
Train Epoch: 405 Loss= 0.346681118 Accuracy= 0.9104
Train Epoch: 406 Loss= 0.346817166 Accuracy= 0.9100
Train Epoch: 407 Loss= 0.346650332 Accuracy= 0.9102
Train Epoch: 408 Loss= 0.346380711 Accuracy= 0.9116
Train Epoch: 409 Loss= 0.346278042 Accuracy= 0.9116
Train Epoch: 410 Loss= 0.346026599 Accuracy= 0.9106
Train Epoch: 411 Loss= 0.345805764 Accuracy= 0.9102
Train Epoch: 412 Loss= 0.345602989 Accuracy= 0.9100
Train Epoch: 413 Loss= 0.345262617 Accuracy= 0.9100
Train Epoch: 414 Loss= 0.345551938 Accuracy= 0.9114
Train Epoch: 415 Loss= 0.345476508 Accuracy= 0.9114
Train Epoch: 416 Loss= 0.345072389 Accuracy= 0.9110
Train Epoch: 417 Loss= 0.345159948 Accuracy= 0.9100
Train Epoch: 418 Loss= 0.344624996 Accuracy= 0.9112
Train Epoch: 419 Loss= 0.344826609 Accuracy= 0.9112
Train Epoch: 420 Loss= 0.344251603 Accuracy= 0.9098
Train Epoch: 421 Loss= 0.344352007 Accuracy= 0.9110
Train Epoch: 422 Loss= 0.344260216 Accuracy= 0.9110
Train Epoch: 423 Loss= 0.343791753 Accuracy= 0.9102
Train Epoch: 424 Loss= 0.343953133 Accuracy= 0.9108
Train Epoch: 425 Loss= 0.343585521 Accuracy= 0.9108
Train Epoch: 426 Loss= 0.343450099 Accuracy= 0.9106
Train Epoch: 427 Loss= 0.343184948 Accuracy= 0.9112
Train Epoch: 428 Loss= 0.343215674 Accuracy= 0.9110
Train Epoch: 429 Loss= 0.343070745 Accuracy= 0.9108
Train Epoch: 430 Loss= 0.342824966 Accuracy= 0.9110
Train Epoch: 431 Loss= 0.342438400 Accuracy= 0.9114
Train Epoch: 432 Loss= 0.342611969 Accuracy= 0.9114
Train Epoch: 433 Loss= 0.342214018 Accuracy= 0.9110
Train Epoch: 434 Loss= 0.342429280 Accuracy= 0.9108
Train Epoch: 435 Loss= 0.341949135 Accuracy= 0.9112
Train Epoch: 436 Loss= 0.342126906 Accuracy= 0.9110
Train Epoch: 437 Loss= 0.341858774 Accuracy= 0.9110
Train Epoch: 438 Loss= 0.341584474 Accuracy= 0.9110
Train Epoch: 439 Loss= 0.341345459 Accuracy= 0.9112
Train Epoch: 440 Loss= 0.341049880 Accuracy= 0.9120
Train Epoch: 441 Loss= 0.340980768 Accuracy= 0.9116
Train Epoch: 442 Loss= 0.340953112 Accuracy= 0.9114
Train Epoch: 443 Loss= 0.340954691 Accuracy= 0.9116
Train Epoch: 444 Loss= 0.340899229 Accuracy= 0.9110
Train Epoch: 445 Loss= 0.340938568 Accuracy= 0.9110
Train Epoch: 446 Loss= 0.341198325 Accuracy= 0.9110
Train Epoch: 447 Loss= 0.340240568 Accuracy= 0.9112
Train Epoch: 448 Loss= 0.340235293 Accuracy= 0.9112
Train Epoch: 449 Loss= 0.340209186 Accuracy= 0.9114
Train Epoch: 450 Loss= 0.339898020 Accuracy= 0.9116
Train Epoch: 451 Loss= 0.339793384 Accuracy= 0.9114
Train Epoch: 452 Loss= 0.339602858 Accuracy= 0.9120
Train Epoch: 453 Loss= 0.339292139 Accuracy= 0.9116
Train Epoch: 454 Loss= 0.339383274 Accuracy= 0.9120
Train Epoch: 455 Loss= 0.339096963 Accuracy= 0.9114
Train Epoch: 456 Loss= 0.339157701 Accuracy= 0.9114
Train Epoch: 457 Loss= 0.338972569 Accuracy= 0.9120
Train Epoch: 458 Loss= 0.338898569 Accuracy= 0.9118
Train Epoch: 459 Loss= 0.338588566 Accuracy= 0.9120
Train Epoch: 460 Loss= 0.338383496 Accuracy= 0.9116
Train Epoch: 461 Loss= 0.338389367 Accuracy= 0.9118
Train Epoch: 462 Loss= 0.338129699 Accuracy= 0.9120
Train Epoch: 463 Loss= 0.338002205 Accuracy= 0.9120
Train Epoch: 464 Loss= 0.337737024 Accuracy= 0.9118
Train Epoch: 465 Loss= 0.337787151 Accuracy= 0.9116
Train Epoch: 466 Loss= 0.337848932 Accuracy= 0.9118
Train Epoch: 467 Loss= 0.337492615 Accuracy= 0.9112
Train Epoch: 468 Loss= 0.337512612 Accuracy= 0.9120
Train Epoch: 469 Loss= 0.337134123 Accuracy= 0.9116
Train Epoch: 470 Loss= 0.336997300 Accuracy= 0.9118
Train Epoch: 471 Loss= 0.336745471 Accuracy= 0.9120
Train Epoch: 472 Loss= 0.337093860 Accuracy= 0.9124
Train Epoch: 473 Loss= 0.336436093 Accuracy= 0.9120
Train Epoch: 474 Loss= 0.336423337 Accuracy= 0.9116
Train Epoch: 475 Loss= 0.336435258 Accuracy= 0.9116
Train Epoch: 476 Loss= 0.336220562 Accuracy= 0.9120
Train Epoch: 477 Loss= 0.336083412 Accuracy= 0.9112
Train Epoch: 478 Loss= 0.336012244 Accuracy= 0.9116
Train Epoch: 479 Loss= 0.335907310 Accuracy= 0.9116
Train Epoch: 480 Loss= 0.336151212 Accuracy= 0.9126
Train Epoch: 481 Loss= 0.335842371 Accuracy= 0.9114
Train Epoch: 482 Loss= 0.335602969 Accuracy= 0.9126
Train Epoch: 483 Loss= 0.335610300 Accuracy= 0.9124
Train Epoch: 484 Loss= 0.335590661 Accuracy= 0.9120
Train Epoch: 485 Loss= 0.335201710 Accuracy= 0.9124
Train Epoch: 486 Loss= 0.334876269 Accuracy= 0.9124
Train Epoch: 487 Loss= 0.334915966 Accuracy= 0.9120
Train Epoch: 488 Loss= 0.335007548 Accuracy= 0.9124
Train Epoch: 489 Loss= 0.334643126 Accuracy= 0.9124
Train Epoch: 490 Loss= 0.334648579 Accuracy= 0.9124
Train Epoch: 491 Loss= 0.334624171 Accuracy= 0.9126
Train Epoch: 492 Loss= 0.334362388 Accuracy= 0.9134
Train Epoch: 493 Loss= 0.334085405 Accuracy= 0.9132
Train Epoch: 494 Loss= 0.333826602 Accuracy= 0.9128
Train Epoch: 495 Loss= 0.333956271 Accuracy= 0.9128
Train Epoch: 496 Loss= 0.334147930 Accuracy= 0.9126
Train Epoch: 497 Loss= 0.333745390 Accuracy= 0.9128
Train Epoch: 498 Loss= 0.333584398 Accuracy= 0.9130
Train Epoch: 499 Loss= 0.333650619 Accuracy= 0.9126
Train Epoch: 500 Loss= 0.333045900 Accuracy= 0.9126
Train Finshied!
Test Accurary: 0.9094

学习效果极好：
result_prediction

Neo Sun