COGAN

发表于 | 分类于 ， | | 本文总阅读数 次
| 字数统计: 1.9k | 阅读时长 ≈ 10
COGAN

COGAN

背景介绍

  COGAN(Coupled Generative Adversarial Networks, 耦合生成式对抗网络):于2016年发表在NIPS上，只有两个以上模型才能称之为耦合，因此COGAN中存在两个生成模型和两个判别模型，而且共用某些网络层，实现耦合效果。

cogan

COGAN特点

  引入两个GAN模型，并且共用某些网络层，实现少参数多功能的效果。

COGAN图像分析

generator
discriminator
discriminator

TensorFlow2.0实现

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
import os
import numpy as np
import cv2 as cv
from functools import reduce
import tensorflow as tf
import tensorflow.keras as keras


def compose(*funcs):
    if funcs:
        return reduce(lambda f, g: lambda *a, **kw: g(f(*a, **kw)), funcs)
    else:
        raise ValueError('Composition of empty sequence not supported.')


def generator(input_shape):
    input_tensor = keras.layers.Input(input_shape, name='input')
    x = input_tensor

    x = compose(keras.layers.Dense(1568, activation='relu', name='dense_relu'),
                keras.layers.Reshape((7, 7, 32), name='reshape'),
                keras.layers.Conv2D(64, (3, 3), (1, 1), 'same', name='conv1'),
                keras.layers.BatchNormalization(momentum=0.8, name='bn1'),
                keras.layers.ReLU(name='relu1'),
                keras.layers.UpSampling2D((2, 2), name='upsampling1'),
                keras.layers.Conv2D(128, (3, 3), (1, 1), 'same', name='conv2'),
                keras.layers.BatchNormalization(momentum=0.8, name='bn2'),
                keras.layers.ReLU(name='relu2'),
                keras.layers.UpSampling2D((2, 2), name='upsampling2'),
                keras.layers.Conv2D(128, (3, 3), (1, 1), 'same', name='conv3'),
                keras.layers.BatchNormalization(momentum=0.8, name='bn3'),
                keras.layers.ReLU(name='relu3'))(x)

    x1 = compose(keras.layers.Conv2D(64, (1, 1), (1, 1), 'same', name='conv_part1_4'),
                 keras.layers.BatchNormalization(momentum=0.8, name='bn_part1_4'),
                 keras.layers.ReLU(name='relu_part1_4'),
                 keras.layers.Conv2D(64, (3, 3), (1, 1), 'same', name='conv_part1_5'),
                 keras.layers.BatchNormalization(momentum=0.8, name='bn_part1_5'),
                 keras.layers.ReLU(name='relu_part1_5'),
                 keras.layers.Conv2D(64, (1, 1), (1, 1), 'same', name='conv_part1_6'),
                 keras.layers.BatchNormalization(momentum=0.8, name='bn_part1_6'),
                 keras.layers.ReLU(name='relu_part1_6'),
                 keras.layers.Conv2D(1, (1, 1), (1, 1), 'same', activation='tanh', name='conv_part1_tanh'))(x)

    x2 = compose(keras.layers.Conv2D(64, (1, 1), (1, 1), 'same', name='conv_part2_4'),
                 keras.layers.BatchNormalization(momentum=0.8, name='bn_part2_4'),
                 keras.layers.ReLU(name='relu_part2_4'),
                 keras.layers.Conv2D(64, (3, 3), (1, 1), 'same', name='conv_part2_5'),
                 keras.layers.BatchNormalization(momentum=0.8, name='bn_part2_5'),
                 keras.layers.ReLU(name='relu_part2_5'),
                 keras.layers.Conv2D(64, (1, 1), (1, 1), 'same', name='conv_part2_6'),
                 keras.layers.BatchNormalization(momentum=0.8, name='bn_part2_6'),
                 keras.layers.ReLU(name='relu_part2_6'),
                 keras.layers.Conv2D(1, (1, 1), (1, 1), 'same', activation='tanh', name='conv_part2_tanh'))(x)

    model_part1 = keras.Model(input_tensor, x1, name='COGAN-Generator1')
    model_part2 = keras.Model(input_tensor, x2, name='COGAN-Generator2')

    return model_part1, model_part2


def discriminator(input_shape):
    input_tensor = keras.layers.Input(input_shape, name='input')
    input_tensor1 = keras.layers.Input(input_shape, name='input1')
    input_tensor2 = keras.layers.Input(input_shape, name='input2')

    x = input_tensor

    x = compose(keras.layers.Conv2D(64, (3, 3), (2, 2), 'same', name='conv1'),
                keras.layers.BatchNormalization(momentum=0.8, name='bn1'),
                keras.layers.LeakyReLU(0.2, name='leakyrelu1'),
                keras.layers.Conv2D(128, (3, 3), (2, 2), 'same', name='conv2'),
                keras.layers.BatchNormalization(momentum=0.8, name='bn2'),
                keras.layers.LeakyReLU(0.2, name='leakyrelu2'),
                keras.layers.Conv2D(64, (3, 3), (2, 2), 'same', name='conv3'),
                keras.layers.BatchNormalization(momentum=0.8, name='bn3'),
                keras.layers.LeakyReLU(0.2, name='leakyrelu3'),
                keras.layers.GlobalAveragePooling2D(name='global_averagepool'))(x)

    model = keras.Model(input_tensor, x, name='COGAN-CODiscriminator')

    model.build(input_shape=(28, 28, 1))
    model.summary()
    keras.utils.plot_model(model, 'COGAN-COdiscriminator.png', show_shapes=True, show_layer_names=True)

    x1 = model(input_tensor1)
    x1 = keras.layers.Dense(1, activation='sigmoid', name='dense_part1_sigmoid')(x1)

    model_part1 = keras.Model(input_tensor1, x1, name='COGAN-Discriminator1')

    x2 = model(input_tensor2)
    x2 = keras.layers.Dense(1, activation='sigmoid', name='dense_part2_sigmoid')(x2)

    model_part2 = keras.Model(input_tensor2, x2, name='COGAN-Discriminator2')

    return model_part1, model_part2


def cogan(input_shape, model_g1, model_g2, model_d1, model_d2):
    input_tensor = keras.layers.Input(input_shape)
    x = input_tensor

    x1 = model_g1(x)
    x2 = model_g2(x)

    model_d1.trainable = False
    model_d2.trainable = False

    x1 = model_d1(x1)
    x2 = model_d1(x2)

    model = keras.Model(input_tensor, [x1, x2], name='COGAN')

    return model


def save_picture(image, save_path, picture_num):
    image = ((image + 1) * 127.5).astype(np.uint8)
    image = np.concatenate([image[i * picture_num:(i + 1) * picture_num] for i in range(picture_num)], axis=2)
    image = np.concatenate([image[i] for i in range(picture_num)], axis=0)
    cv.imwrite(save_path, image)


if __name__ == '__main__':
    (x, _), (_, _) = keras.datasets.mnist.load_data()
    batch_size = 256
    epochs = 20
    tf.random.set_seed(22)
    save_path = r'.\cogan'
    if not os.path.exists(save_path):
        os.makedirs(save_path)

    x = x[..., np.newaxis].astype(np.float32) / 127.5 - 1
    x = tf.data.Dataset.from_tensor_slices(x).batch(batch_size)

    optimizer = keras.optimizers.Adam(0.0002, 0.5)
    loss = keras.losses.BinaryCrossentropy()

    real_d1acc = keras.metrics.BinaryAccuracy()
    fake_d1acc = keras.metrics.BinaryAccuracy()
    real_d2acc = keras.metrics.BinaryAccuracy()
    fake_d2acc = keras.metrics.BinaryAccuracy()
    g1acc = keras.metrics.BinaryAccuracy()
    g2acc = keras.metrics.BinaryAccuracy()

    model_d1, model_d2 = discriminator(input_shape=(28, 28, 1))
    model_d1.compile(optimizer=optimizer, loss='binary_crossentropy')
    model_d2.compile(optimizer=optimizer, loss='binary_crossentropy')

    model_g1, model_g2 = generator(input_shape=(100,))

    model_g1.build(input_shape=(100,))
    model_g1.summary()
    keras.utils.plot_model(model_g1, 'COGAN-generator1.png', show_shapes=True, show_layer_names=True)

    model_g2.build(input_shape=(100,))
    model_g2.summary()
    keras.utils.plot_model(model_g2, 'COGAN-generator2.png', show_shapes=True, show_layer_names=True)

    model_d1.build(input_shape=(28, 28, 1))
    model_d1.summary()
    keras.utils.plot_model(model_d1, 'COGAN-discriminator1.png', show_shapes=True, show_layer_names=True)

    model_d2.build(input_shape=(28, 28, 1))
    model_d2.summary()
    keras.utils.plot_model(model_d2, 'COGAN-discriminator2.png', show_shapes=True, show_layer_names=True)

    model = cogan(input_shape=(100,), model_g1=model_g1, model_g2=model_g2, model_d1=model_d1, model_d2=model_d2)
    model.compile(optimizer=optimizer, loss=['binary_crossentropy', 'binary_crossentropy'])

    model.build(input_shape=(100,))
    model.summary()
    keras.utils.plot_model(model, 'COGAN.png', show_shapes=True, show_layer_names=True)

    for epoch in range(epochs):
        x = x.shuffle(np.random.randint(0, 10000))
        x_db = iter(x)

        for step, real_image in enumerate(x_db):
            noise = np.random.normal(0, 1, (real_image.shape[0], 100))
            fake_image1 = model_g1(noise)
            fake_image2 = model_g2(noise)

            real_d1acc(np.ones((real_image.shape[0], 1)), model_d1(real_image))
            fake_d1acc(np.zeros((real_image.shape[0], 1)), model_d1(fake_image1))
            real_d2acc(np.ones((real_image.shape[0], 1)), model_d2(real_image))
            fake_d2acc(np.zeros((real_image.shape[0], 1)), model_d2(fake_image2))
            g1acc(np.ones((real_image.shape[0], 1)), model(noise)[0])
            g2acc(np.ones((real_image.shape[0], 1)), model(noise)[1])

            real_d1loss = model_d1.train_on_batch(real_image, np.ones((real_image.shape[0], 1)))
            fake_d1loss = model_d1.train_on_batch(fake_image1, np.zeros((real_image.shape[0], 1)))
            real_d2loss = model_d2.train_on_batch(real_image, np.ones((real_image.shape[0], 1)))
            fake_d2loss = model_d2.train_on_batch(fake_image2, np.zeros((real_image.shape[0], 1)))
            gloss = model.train_on_batch(noise, [np.ones((real_image.shape[0], 1)), np.ones((real_image.shape[0], 1))])

            if step % 20 == 0:
                print('epoch = {}, step = {}, real_d1acc = {}, fake_d1acc = {}, real_d1acc = {}, fake_d1acc = {}, g1acc = {}, g2acc = {}'.format(epoch, step, real_d1acc.result(), fake_d1acc.result(), real_d2acc.result(), fake_d2acc.result(), g1acc.result(), g2acc.result()))
                real_d1acc.reset_states()
                fake_d1acc.reset_states()
                real_d2acc.reset_states()
                fake_d2acc.reset_states()
                g1acc.reset_states()
                g2acc.reset_states()
                fake_data = np.random.normal(0, 1, (100, 100))
                fake_image = model_g1(fake_data)
                save_picture(fake_image.numpy(), save_path + '\\epoch{}_step{}.jpg'.format(epoch, step), 10)

cogan

模型运行结果

cogan

小技巧

  1. 图像输入可以先将其归一化到0-1之间或者-1-1之间，因为网络的参数一般都比较小，所以归一化后计算方便，收敛较快。
  2. 注意其中的一些维度变换和numpy，tensorflow常用操作，否则在阅读代码时可能会产生一些困难。
  3. 可以设置一些权重的保存方式，学习率的下降方式和早停方式。
  4. COGAN对于网络结构，优化器参数，网络层的一些超参数都是非常敏感的，效果不好不容易发现原因，这可能需要较多的工程实践经验。
  5. 先创建判别器，然后进行compile，这样判别器就固定了，然后创建生成器时，不要训练判别器，需要将判别器的trainable改成False，此时不会影响之前固定的判别器，这个可以通过模型的_collection_collected_trainable_weights属性查看，如果该属性为空，则模型不训练，否则模型可以训练，compile之后，该属性固定，无论后面如何修改trainable，只要不重新compile，都不影响训练。
  6. COGAN中引入了两个GAN网络，其目的不是实现一种功能，虽然在这里我是实现了一种功能，其实他们共用的目的是节约特征提取网络参数，可以让一个GAN来生成某一种图像，另一个GAN来生成另一种图像，如model_g1生成手写数字，model_g2生成镜像手写数字，如果正常训练两个GAN模型，分别生成手写数字和镜像手写数字，同样的网络结构需要1.2M的参数量，而耦合之后的参数量约为0.6M，而且模型越大效果越明显。

COGAN小结

  COGAN是一种有效的耦合生成式对抗网络，从上图可以看出COGAN模型的参数量只有0.6M，是一种可以同时完成多任务的网络模型，小伙伴们一定要掌握它。

-------------本文结束感谢您的阅读-------------
0%