• 动态图机制-DyGraph
    • 设置和基本用法
    • 基于DyGraph构建网络
    • 使用DyGraph训练模型
    • 模型参数的保存
    • 模型评估
    • 编写兼容的模型

    动态图机制-DyGraph

    PaddlePaddle的DyGraph模式是一种动态的图执行机制,可以立即执行结果,无需构建整个图。同时,和以往静态的执行计算图不同,DyGraph模式下您的所有操作可以立即获得执行结果,而不必等待所构建的计算图全部执行完成,这样可以让您更加直观地构建PaddlePaddle下的深度学习任务,以及进行模型的调试,同时还减少了大量用于构建静态计算图的代码,使得您编写、调试网络的过程变得更加便捷。

    PaddlePaddle DyGraph是一个更加灵活易用的模式,可提供:

    • 更加灵活便捷的代码组织结构: 使用python的执行控制流程和面向对象的模型设计

    • 更加便捷的调试功能: 直接使用python的打印方法即时打印所需要的结果,从而检查正在运行的模型结果便于测试更改

    • 和静态执行图通用的模型代码:同样的模型代码可以使用更加便捷的DyGraph调试,执行,同时也支持使用原有的静态图模式执行

    设置和基本用法

    • 升级到最新的PaddlePaddle 1.5:
    1. pip install -q --upgrade paddlepaddle==1.5
    • 使用fluid.dygraph.guard(place=None) 上下文:
    1. import paddle.fluid as fluid
    2. with fluid.dygraph.guard():
    3. # write your executable dygraph code here

    现在您就可以在fluid.dygraph.guard()上下文环境中使用DyGraph的模式运行网络了,DyGraph将改变以往PaddlePaddle的执行方式: 现在他们将会立即执行,并且将计算结果返回给Python。

    Dygraph将非常适合和Numpy一起使用,使用fluid.dygraph.to_variable(x)将会将ndarray转换为fluid.Variable,而使用fluid.Variable.numpy()将可以把任意时刻获取到的计算结果转换为Numpyndarray

    1. x = np.ones([2, 2], np.float32)
    2. with fluid.dygraph.guard():
    3. inputs = []
    4. for _ in range(10):
    5. inputs.append(fluid.dygraph.to_variable(x))
    6. ret = fluid.layers.sums(inputs)
    7. print(ret.numpy())
    8. [[10. 10.]
    9. [10. 10.]]
    10. Process finished with exit code 0

    这里创建了一系列ndarray的输入,执行了一个sum操作之后,我们可以直接将运行的结果打印出来

    然后通过调用reduce_sum后使用Variable.backward()方法执行反向,使用Variable.gradient()方法即可获得反向网络执行完成后的梯度值的ndarray形式:

    1. loss = fluid.layers.reduce_sum(ret)
    2. loss.backward()
    3. print(loss.gradient())

    得到输出 :

    1. [1.]
    2. Process finished with exit code 0

    基于DyGraph构建网络

    • 编写一段用于DyGraph执行的Object-Oriented-Designed, PaddlePaddle模型代码主要由以下三个部分组成: 请注意,如果您设计的这一层结构是包含参数的,则必须要使用继承自fluid.dygraph.Layer的Object-Oriented-Designed的类来描述该层的行为。

      • 建立一个可以在DyGraph模式中执行的,Object-Oriented的网络,需要继承自fluid.dygraph.Layer,其中需要调用基类的init方法,并且实现带有参数namescope(用来标识本层的名字)的_init构造函数,在构造函数中,我们通常会执行一些例如参数初始化,子网络初始化的操作,执行这些操作时不依赖于输入的动态信息:
    1. class MyLayer(fluid.dygraph.Layer):
    2. def __init__(self, name_scope):
    3. super(MyLayer, self).__init__(name_scope)
    • 实现一个forward(self, *inputs)的执行函数,该函数将负责执行实际运行时网络的执行逻辑, 该函数将会在每一轮训练/预测中被调用,这里我们将执行一个简单的relu -> elementwise add -> reduce sum
    1. def forward(self, inputs):
    2. x = fluid.layers.relu(inputs)
    3. self._x_for_debug = x
    4. x = fluid.layers.elementwise_mul(x, x)
    5. x = fluid.layers.reduce_sum(x)
    6. return [x]
    • fluid.dygraph.guard()中执行:

      • 使用Numpy构建输入:
    1. np_inp = np.array([1.0, 2.0, -1.0], dtype=np.float32)
    • 转换输入的ndarrayVariable, 并执行前向网络获取返回值: 使用fluid.dygraph.to_variable(np_inp)转换Numpy输入为DyGraph接收的输入,然后使用my_layer(var_inp)[0]调用callable object并且获取了x作为返回值,利用x.numpy()方法直接获取了执行得到的xndarray返回值。
    1. with fluid.dygraph.guard():
    2. var_inp = fluid.dygraph.to_variable(np_inp)
    3. my_layer = MyLayer("my_layer")
    4. x = my_layer(var_inp)[0]
    5. dy_out = x.numpy()
    • 计算梯度:自动微分对于实现机器学习算法(例如用于训练神经网络的反向传播)来说很有用, 使用x.backward()方法可以从某个fluid.Varaible开始执行反向网络,同时利用my_layer._x_for_debug.gradient()获取了网络中x梯度的ndarray 返回值:
    1. x.backward()
    2. dy_grad = my_layer._x_for_debug.gradient()

    完整代码如下:

    1. import paddle.fluid as fluid
    2. import numpy as np
    3. class MyLayer(fluid.dygraph.Layer):
    4. def __init__(self, name_scope):
    5. super(MyLayer, self).__init__(name_scope)
    6. def forward(self, inputs):
    7. x = fluid.layers.relu(inputs)
    8. self._x_for_debug = x
    9. x = fluid.layers.elementwise_mul(x, x)
    10. x = fluid.layers.reduce_sum(x)
    11. return [x]
    12. if __name__ == '__main__':
    13. np_inp = np.array([1.0, 2.0, -1.0], dtype=np.float32)
    14. with fluid.dygraph.guard():
    15. var_inp = fluid.dygraph.to_variable(np_inp)
    16. my_layer = MyLayer("my_layer")
    17. x = my_layer(var_inp)[0]
    18. dy_out = x.numpy()
    19. x.backward()
    20. dy_grad = my_layer._x_for_debug.gradient()
    21. my_layer.clear_gradients() # 将参数梯度清零以保证下一轮训练的正确性

    使用DyGraph训练模型

    接下来我们将以“手写数字识别”这个最基础的模型为例,展示如何利用DyGraph模式搭建并训练一个模型:

    有关手写数字识别的相关理论知识请参考PaddleBook中的内容,我们在这里默认您已经了解了该模型所需的深度学习理论知识。

    • 准备数据,我们使用paddle.dataset.mnist作为训练所需要的数据集:
    1. train_reader = paddle.batch(
    2. paddle.dataset.mnist.train(), batch_size=BATCH_SIZE, drop_last=True)
    • 构建网络,虽然您可以根据之前的介绍自己定义所有的网络结构,但是您也可以直接使用fluid.dygraph.Layer当中我们为您定制好的一些基础网络结构,这里我们利用fluid.dygraph.Conv2D以及fluid.dygraph.Pool2d构建了基础的SimpleImgConvPool
    1. class SimpleImgConvPool(fluid.dygraph.Layer):
    2. def __init__(self,
    3. name_scope,
    4. num_filters,
    5. filter_size,
    6. pool_size,
    7. pool_stride,
    8. pool_padding=0,
    9. pool_type='max',
    10. global_pooling=False,
    11. conv_stride=1,
    12. conv_padding=0,
    13. conv_dilation=1,
    14. conv_groups=1,
    15. act=None,
    16. use_cudnn=False,
    17. param_attr=None,
    18. bias_attr=None):
    19. super(SimpleImgConvPool, self).__init__(name_scope)
    20. self._conv2d = fluid.dygraph.Conv2D(
    21. self.full_name(),
    22. num_filters=num_filters,
    23. filter_size=filter_size,
    24. stride=conv_stride,
    25. padding=conv_padding,
    26. dilation=conv_dilation,
    27. groups=conv_groups,
    28. param_attr=None,
    29. bias_attr=None,
    30. act=act,
    31. use_cudnn=use_cudnn)
    32. self._pool2d = fluid.dygraph.Pool2D(
    33. self.full_name(),
    34. pool_size=pool_size,
    35. pool_type=pool_type,
    36. pool_stride=pool_stride,
    37. pool_padding=pool_padding,
    38. global_pooling=global_pooling,
    39. use_cudnn=use_cudnn)
    40. def forward(self, inputs):
    41. x = self._conv2d(inputs)
    42. x = self._pool2d(x)
    43. return x

    注意: 构建网络时子网络的定义和使用请在init中进行, 而子网络的执行则在forward函数中进行

    • 利用已经构建好的SimpleImgConvPool组成最终的MNIST网络:
    1. class MNIST(fluid.dygraph.Layer):
    2. def __init__(self, name_scope):
    3. super(MNIST, self).__init__(name_scope)
    4. self._simple_img_conv_pool_1 = SimpleImgConvPool(
    5. self.full_name(), 20, 5, 2, 2, act="relu")
    6. self._simple_img_conv_pool_2 = SimpleImgConvPool(
    7. self.full_name(), 50, 5, 2, 2, act="relu")
    8. pool_2_shape = 50 * 4 * 4
    9. SIZE = 10
    10. scale = (2.0 / (pool_2_shape**2 * SIZE))**0.5
    11. self._fc = fluid.dygraph.FC(self.full_name(),
    12. 10,
    13. param_attr=fluid.param_attr.ParamAttr(
    14. initializer=fluid.initializer.NormalInitializer(
    15. loc=0.0, scale=scale)),
    16. act="softmax")
    17. def forward(self, inputs, label=None):
    18. x = self._simple_img_conv_pool_1(inputs)
    19. x = self._simple_img_conv_pool_2(x)
    20. x = self._fc(x)
    21. if label is not None:
    22. acc = fluid.layers.accuracy(input=x, label=label)
    23. return x, acc
    24. else:
    25. return x
    • fluid.dygraph.guard()中定义配置好的MNIST网络结构,此时即使没有训练也可以在fluid.dygraph.guard()中调用模型并且检查输出:
    1. with fluid.dygraph.guard():
    2. mnist = MNIST("mnist")
    3. id, data = list(enumerate(train_reader()))[0]
    4. dy_x_data = np.array(
    5. [x[0].reshape(1, 28, 28)
    6. for x in data]).astype('float32')
    7. img = fluid.dygraph.to_variable(dy_x_data)
    8. print("result is: {}".format(mnist(img).numpy()))
    9. result is: [[0.10135901 0.1051138 0.1027941 ... 0.0972859 0.10221873 0.10165327]
    10. [0.09735426 0.09970362 0.10198303 ... 0.10134517 0.10179105 0.10025002]
    11. [0.09539858 0.10213123 0.09543551 ... 0.10613529 0.10535969 0.097991 ]
    12. ...
    13. [0.10120598 0.0996111 0.10512722 ... 0.10067689 0.10088114 0.10071224]
    14. [0.09889644 0.10033772 0.10151272 ... 0.10245881 0.09878646 0.101483 ]
    15. [0.09097178 0.10078511 0.10198414 ... 0.10317434 0.10087223 0.09816764]]
    16. Process finished with exit code 0
    • 构建训练循环,在每一轮参数更新完成后我们调用mnist.clear_gradients()来重置梯度:
    1. with fluid.dygraph.guard():
    2. epoch_num = 5
    3. BATCH_SIZE = 64
    4. train_reader = paddle.batch(
    5. paddle.dataset.mnist.train(), batch_size=32, drop_last=True)
    6. mnist = MNIST("mnist")
    7. id, data = list(enumerate(train_reader()))[0]
    8. adam = fluid.optimizer.AdamOptimizer(learning_rate=0.001)
    9. for epoch in range(epoch_num):
    10. for batch_id, data in enumerate(train_reader()):
    11. dy_x_data = np.array([x[0].reshape(1, 28, 28)
    12. for x in data]).astype('float32')
    13. y_data = np.array(
    14. [x[1] for x in data]).astype('int64').reshape(-1, 1)
    15. img = fluid.dygraph.to_variable(dy_x_data)
    16. label = fluid.dygraph.to_variable(y_data)
    17. cost = mnist(img)
    18. loss = fluid.layers.cross_entropy(cost, label)
    19. avg_loss = fluid.layers.mean(loss)
    20. if batch_id % 100 == 0 and batch_id is not 0:
    21. print("epoch: {}, batch_id: {}, loss is: {}".format(epoch, batch_id, avg_loss.numpy()))
    22. avg_loss.backward()
    23. adam.minimize(avg_loss)
    24. mnist.clear_gradients()
    • 变量及优化器

    模型的参数或者任何您希望检测的值可以作为变量封装在类中,然后通过对象获取并使用numpy()方法获取其ndarray的输出, 在训练过程中您可以使用mnist.parameters()来获取到网络中所有的参数,也可以指定某一个Layer的某个参数或者parameters()来获取该层的所有参数,使用numpy()方法随时查看参数的值

    反向运行后调用之前定义的Adam优化器对象的minimize方法进行参数更新:

    1. with fluid.dygraph.guard():
    2. epoch_num = 5
    3. BATCH_SIZE = 64
    4. mnist = MNIST("mnist")
    5. adam = fluid.optimizer.AdamOptimizer(learning_rate=0.001)
    6. train_reader = paddle.batch(
    7. paddle.dataset.mnist.train(), batch_size= BATCH_SIZE, drop_last=True)
    8. np.set_printoptions(precision=3, suppress=True)
    9. for epoch in range(epoch_num):
    10. for batch_id, data in enumerate(train_reader()):
    11. dy_x_data = np.array(
    12. [x[0].reshape(1, 28, 28)
    13. for x in data]).astype('float32')
    14. y_data = np.array(
    15. [x[1] for x in data]).astype('int64').reshape(BATCH_SIZE, 1)
    16. img = fluid.dygraph.to_variable(dy_x_data)
    17. label = fluid.dygraph.to_variable(y_data)
    18. label.stop_gradient = True
    19. cost = mnist(img)
    20. loss = fluid.layers.cross_entropy(cost, label)
    21. avg_loss = fluid.layers.mean(loss)
    22. dy_out = avg_loss.numpy()
    23. avg_loss.backward()
    24. adam.minimize(avg_loss)
    25. mnist.clear_gradients()
    26. dy_param_value = {}
    27. for param in mnist.parameters():
    28. dy_param_value[param.name] = param.numpy()
    29. if batch_id % 20 == 0:
    30. print("Loss at step {}: {}".format(batch_id, avg_loss.numpy()))
    31. print("Final loss: {}".format(avg_loss.numpy()))
    32. print("_simple_img_conv_pool_1_conv2d W's mean is: {}".format(mnist._simple_img_conv_pool_1._conv2d._filter_param.numpy().mean()))
    33. print("_simple_img_conv_pool_1_conv2d Bias's mean is: {}".format(mnist._simple_img_conv_pool_1._conv2d._bias_param.numpy().mean()))
    34. Loss at step 0: [2.302]
    35. Loss at step 20: [1.616]
    36. Loss at step 40: [1.244]
    37. Loss at step 60: [1.142]
    38. Loss at step 80: [0.911]
    39. Loss at step 100: [0.824]
    40. Loss at step 120: [0.774]
    41. Loss at step 140: [0.626]
    42. Loss at step 160: [0.609]
    43. Loss at step 180: [0.627]
    44. Loss at step 200: [0.466]
    45. Loss at step 220: [0.499]
    46. Loss at step 240: [0.614]
    47. Loss at step 260: [0.585]
    48. Loss at step 280: [0.503]
    49. Loss at step 300: [0.423]
    50. Loss at step 320: [0.509]
    51. Loss at step 340: [0.348]
    52. Loss at step 360: [0.452]
    53. Loss at step 380: [0.397]
    54. Loss at step 400: [0.54]
    55. Loss at step 420: [0.341]
    56. Loss at step 440: [0.337]
    57. Loss at step 460: [0.155]
    58. Final loss: [0.164]
    59. _simple_img_conv_pool_1_conv2d W's mean is: 0.00606656912714
    60. _simple_img_conv_pool_1_conv2d Bias's mean is: -3.4576318285e-05
    • 性能

    在使用fluid.dygraph.guard()时可以通过传入fluid.CUDAPlace(0)或者fluid.CPUPlace()来选择执行DyGraph的设备,通常如果不做任何处理将会自动适配您的设备。

    模型参数的保存

    在模型训练中可以使用fluid.dygraph.save_persistables(your_model_object.state_dict(), "save_dir", optimizers=None)来保存your_model_object中所有的模型参数, 以及使用learning rate decay的优化器。也可以自定义需要保存的“参数名” - “参数对象”的Python Dictionary传入。

    同样可以使用models,optimizers = fluid.dygraph.load_persistables("save_dir")获取保存的模型参数和优化器。

    再使用your_modle_object.load_dict(models)接口来恢复保存的模型参数从而达到继续训练的目的。

    以及使用your_optimizer_object.load(optimizers)接口来恢复保存的优化器中的learning rate decay

    下面的代码展示了如何在“手写数字识别”任务中保存参数并且读取已经保存的参数来继续训练。

    1. with fluid.dygraph.guard():
    2. epoch_num = 5
    3. BATCH_SIZE = 64
    4. mnist = MNIST("mnist")
    5. adam = fluid.optimizer.Adam(learning_rate=0.001)
    6. train_reader = paddle.batch(
    7. paddle.dataset.mnist.train(), batch_size= BATCH_SIZE, drop_last=True)
    8. np.set_printoptions(precision=3, suppress=True)
    9. dy_param_init_value={}
    10. for epoch in range(epoch_num):
    11. for batch_id, data in enumerate(train_reader()):
    12. dy_x_data = np.array(
    13. [x[0].reshape(1, 28, 28)
    14. for x in data]).astype('float32')
    15. y_data = np.array(
    16. [x[1] for x in data]).astype('int64').reshape(BATCH_SIZE, 1)
    17. img = fluid.dygraph.to_variable(dy_x_data)
    18. label = fluid.dygraph.to_variable(y_data)
    19. label.stop_gradient = True
    20. cost = mnist(img)
    21. loss = fluid.layers.cross_entropy(cost, label)
    22. avg_loss = fluid.layers.mean(loss)
    23. dy_out = avg_loss.numpy()
    24. avg_loss.backward()
    25. adam.minimize(avg_loss)
    26. if batch_id == 20:
    27. fluid.dygraph.save_persistables(mnist.state_dict(), "save_dir", adam)
    28. mnist.clear_gradients()
    29. if batch_id == 20:
    30. for param in mnist.parameters():
    31. dy_param_init_value[param.name] = param.numpy()
    32. model, _ = fluid.dygraph.load_persistables("save_dir")
    33. mnist.load_dict(model)
    34. break
    35. if epoch == 0:
    36. break
    37. restore = mnist.parameters()
    38. # check save and load
    39. success = True
    40. for value in restore:
    41. if (not np.array_equal(value.numpy(), dy_param_init_value[value.name])) or (not np.isfinite(value.numpy().all())) or (np.isnan(value.numpy().any())):
    42. success = False
    43. print("model save and load success? {}".format(success))

    模型评估

    当我们需要在DyGraph模式下利用搭建的模型进行预测任务,可以使用YourModel.eval()接口,在之前的手写数字识别模型中我们使用mnist.eval()来启动预测模式(我们默认在fluid.dygraph.guard()上下文中是训练模式),在预测的模式下,DyGraph将只会执行前向的预测网络,而不会进行自动求导并执行反向网络:

    下面的代码展示了如何使用DyGraph模式训练一个用于执行“手写数字识别”任务的模型并保存,并且利用已经保存好的模型进行预测。

    我们在fluid.dygraph.guard()上下文中进行了模型的保存和训练,值得注意的是,当我们需要在训练的过程中进行预测时需要使用YourModel.eval()切换到预测模式,并且在预测完成后使用YourModel.train()切换回训练模式继续训练。

    我们在inference_mnist中启用另一个fluid.dygraph.guard(),并在其上下文中load之前保存的checkpoint进行预测,同样的在执行预测前需要使用YourModel.eval()来切换到预测模式。

    1. def test_mnist(reader, model, batch_size):
    2. acc_set = []
    3. avg_loss_set = []
    4. for batch_id, data in enumerate(reader()):
    5. dy_x_data = np.array([x[0].reshape(1, 28, 28)
    6. for x in data]).astype('float32')
    7. y_data = np.array(
    8. [x[1] for x in data]).astype('int64').reshape(batch_size, 1)
    9. img = fluid.dygraph.to_variable(dy_x_data)
    10. label = fluid.dygraph.to_variable(y_data)
    11. label.stop_gradient = True
    12. prediction, acc = model(img, label)
    13. loss = fluid.layers.cross_entropy(input=prediction, label=label)
    14. avg_loss = fluid.layers.mean(loss)
    15. acc_set.append(float(acc.numpy()))
    16. avg_loss_set.append(float(avg_loss.numpy()))
    17. # get test acc and loss
    18. acc_val_mean = np.array(acc_set).mean()
    19. avg_loss_val_mean = np.array(avg_loss_set).mean()
    20. return avg_loss_val_mean, acc_val_mean
    21. def inference_mnist():
    22. with fluid.dygraph.guard():
    23. mnist_infer = MNIST("mnist")
    24. # load checkpoint
    25. model_dict, _ = fluid.dygraph.load_persistables("save_dir")
    26. mnist_infer.load_dict(model_dict)
    27. print("checkpoint loaded")
    28. # start evaluate mode
    29. mnist_infer.eval()
    30. def load_image(file):
    31. im = Image.open(file).convert('L')
    32. im = im.resize((28, 28), Image.ANTIALIAS)
    33. im = np.array(im).reshape(1, 1, 28, 28).astype(np.float32)
    34. im = im / 255.0 * 2.0 - 1.0
    35. return im
    36. cur_dir = os.path.dirname(os.path.realpath(__file__))
    37. tensor_img = load_image(cur_dir + '/image/2.png')
    38. results = mnist_infer(fluid.dygraph.to_variable(tensor_img))
    39. lab = np.argsort(results.numpy())
    40. print("Inference result of image/infer_3.png is: %d" % lab[0][-1])
    41. with fluid.dygraph.guard():
    42. epoch_num = 1
    43. BATCH_SIZE = 64
    44. mnist = MNIST("mnist")
    45. adam = fluid.optimizer.AdamOptimizer(learning_rate=0.001)
    46. test_reader = paddle.batch(
    47. paddle.dataset.mnist.test(), batch_size=BATCH_SIZE, drop_last=True)
    48. train_reader = paddle.batch(
    49. paddle.dataset.mnist.train(),
    50. batch_size=BATCH_SIZE,
    51. drop_last=True)
    52. for epoch in range(epoch_num):
    53. for batch_id, data in enumerate(train_reader()):
    54. dy_x_data = np.array([x[0].reshape(1, 28, 28)
    55. for x in data]).astype('float32')
    56. y_data = np.array(
    57. [x[1] for x in data]).astype('int64').reshape(-1, 1)
    58. img = fluid.dygraph.to_variable(dy_x_data)
    59. label = fluid.dygraph.to_variable(y_data)
    60. label.stop_gradient = True
    61. cost, acc = mnist(img, label)
    62. loss = fluid.layers.cross_entropy(cost, label)
    63. avg_loss = fluid.layers.mean(loss)
    64. avg_loss.backward()
    65. adam.minimize(avg_loss)
    66. # save checkpoint
    67. mnist.clear_gradients()
    68. if batch_id % 100 == 0:
    69. print("Loss at epoch {} step {}: {:}".format(
    70. epoch, batch_id, avg_loss.numpy()))
    71. mnist.eval()
    72. test_cost, test_acc = test_mnist(test_reader, mnist, BATCH_SIZE)
    73. mnist.train()
    74. print("Loss at epoch {} , Test avg_loss is: {}, acc is: {}".format(
    75. epoch, test_cost, test_acc))
    76. fluid.dygraph.save_persistables(mnist.state_dict(), "save_dir")
    77. print("checkpoint saved")
    78. inference_mnist()
    79. Loss at epoch 0 step 0: [2.2991252]
    80. Loss at epoch 0 step 100: [0.15491392]
    81. Loss at epoch 0 step 200: [0.13315125]
    82. Loss at epoch 0 step 300: [0.10253005]
    83. Loss at epoch 0 step 400: [0.04266362]
    84. Loss at epoch 0 step 500: [0.08894891]
    85. Loss at epoch 0 step 600: [0.08999012]
    86. Loss at epoch 0 step 700: [0.12975612]
    87. Loss at epoch 0 step 800: [0.15257305]
    88. Loss at epoch 0 step 900: [0.07429226]
    89. Loss at epoch 0 , Test avg_loss is: 0.05995981965082674, acc is: 0.9794671474358975
    90. checkpoint saved
    91. No optimizer loaded. If you didn't save optimizer, please ignore this. The program can still work with new optimizer.
    92. checkpoint loaded
    93. Inference result of image/infer_3.png is: 3

    编写兼容的模型

    以上一步中手写数字识别的例子为例,动态图的模型代码可以直接用于静态图中作为模型代码,执行时,直接使用PaddlePaddle静态图执行方式即可,这里以静态图中的executor为例, 模型代码可以直接使用之前的模型代码,执行时使用Executor执行即可

    1. epoch_num = 1
    2. BATCH_SIZE = 64
    3. exe = fluid.Executor(fluid.CPUPlace())
    4. mnist = MNIST("mnist")
    5. sgd = fluid.optimizer.SGDOptimizer(learning_rate=1e-3)
    6. train_reader = paddle.batch(
    7. paddle.dataset.mnist.train(), batch_size=BATCH_SIZE, drop_last=True)
    8. img = fluid.layers.data(
    9. name='pixel', shape=[1, 28, 28], dtype='float32')
    10. label = fluid.layers.data(name='label', shape=[1], dtype='int64')
    11. cost = mnist(img)
    12. loss = fluid.layers.cross_entropy(cost, label)
    13. avg_loss = fluid.layers.mean(loss)
    14. sgd.minimize(avg_loss)
    15. out = exe.run(fluid.default_startup_program())
    16. for epoch in range(epoch_num):
    17. for batch_id, data in enumerate(train_reader()):
    18. static_x_data = np.array(
    19. [x[0].reshape(1, 28, 28)
    20. for x in data]).astype('float32')
    21. y_data = np.array(
    22. [x[1] for x in data]).astype('int64').reshape([BATCH_SIZE, 1])
    23. fetch_list = [avg_loss.name]
    24. out = exe.run(
    25. fluid.default_main_program(),
    26. feed={"pixel": static_x_data,
    27. "label": y_data},
    28. fetch_list=fetch_list)
    29. static_out = out[0]
    30. if batch_id % 100 == 0 and batch_id is not 0:
    31. print("epoch: {}, batch_id: {}, loss: {}".format(epoch, batch_id, static_out))