对于Tensorflow中的优化器（Optimizer），目前已有的有以下：

• tf.train.GradientDescentOptimizer()? ? ? ?常用的梯度下降法
• tf.train.Adadeltatimizer()
• tf.train.Adagradoptimizer()
• tf.train.AdagradDAOptimizer()
• tf.train.MomentumOptimizer()
• tf.train.AdamOptimizer()
• tf.train.Ftrloptimizer()
• tf.train.ProximalGradientDescentOptimizer()
• tf.train.ProximalAdgradOptimizer()
• tf.train.Rmspropoptimizer()

不同的优化器有各自的特点，不能说谁好谁坏，有的收敛速度慢，有的收敛速度快。

?

此处以MNIST数据集识别分类为例进行不同优化器的测试

1、梯度下降法：tf.train.GradientDescentOptimizer()?

?

训练测试结果：

Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
Iter 0,Testing Accuracy 0.8237
Iter 1,Testing Accuracy 0.8902
Iter 2,Testing Accuracy 0.8997
Iter 3,Testing Accuracy 0.9048
Iter 4,Testing Accuracy 0.9089
Iter 5,Testing Accuracy 0.9107
Iter 6,Testing Accuracy 0.9119
Iter 7,Testing Accuracy 0.914
Iter 8,Testing Accuracy 0.9148
Iter 9,Testing Accuracy 0.9161
Iter 10,Testing Accuracy 0.9173
Iter 11,Testing Accuracy 0.9182
Iter 12,Testing Accuracy 0.9203
Iter 13,Testing Accuracy 0.9199
Iter 14,Testing Accuracy 0.9192
Iter 15,Testing Accuracy 0.9196
Iter 16,Testing Accuracy 0.9196
Iter 17,Testing Accuracy 0.9207
Iter 18,Testing Accuracy 0.9214
Iter 19,Testing Accuracy 0.9209
Iter 20,Testing Accuracy 0.9217

?

?

2、tf.train.AdamOptimizer()? ? 优化器的使用

即将

换成

?

训练测试结果：

Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
Iter 0,Testing Accuracy 0.917
Iter 1,Testing Accuracy 0.9242
Iter 2,Testing Accuracy 0.9287
Iter 3,Testing Accuracy 0.9287
Iter 4,Testing Accuracy 0.9285
Iter 5,Testing Accuracy 0.9317
Iter 6,Testing Accuracy 0.931
Iter 7,Testing Accuracy 0.9308
Iter 8,Testing Accuracy 0.9325
Iter 9,Testing Accuracy 0.929
Iter 10,Testing Accuracy 0.9293
Iter 11,Testing Accuracy 0.933
Iter 12,Testing Accuracy 0.9282
Iter 13,Testing Accuracy 0.9297
Iter 14,Testing Accuracy 0.9323
Iter 15,Testing Accuracy 0.9309
Iter 16,Testing Accuracy 0.9296
Iter 17,Testing Accuracy 0.9317
Iter 18,Testing Accuracy 0.9304
Iter 19,Testing Accuracy 0.9281
Iter 20,Testing Accuracy 0.9314

?

可见，在MNIST数据下，使用设计的神经网络，此种优化器性能较好。

?