MXNet (6) Parameter and Block Naming (下)

上一篇我們示範了 Name scope 會將 parent 的 prefix 加入 children 的命名之中。

今天來看看 nested block 的命名如何運作。

1. 快速 recall:

建立一個簡單的網路架構，並利用 parent 的 prefix 加入 children 的命名之中。

class SimpleNet(nn.HybridBlock):

    def __init__(self, **kargs):
        super(SimpleNet, self).__init__(**kargs)
        with self.name_scope():
            self.body = nn.HybridSequential()
            self.body.add(
                nn.Conv2D(20, 3),
                nn.Conv2D(36, 3),
            )

    def hybrid_forward(self, F, x, *args, **kwargs):
        return self.body(x)

實例化及初始化:

net = SimpleNet()
net.initialize()
print(net.collect_params())

children 的 key 包含 parent 類的 simplenet0_。

simplenet0_ (
  Parameter simplenet0_conv0_weight (shape=(20, 0, 3, 3), dtype=)
  Parameter simplenet0_conv0_bias (shape=(20,), dtype=)
  Parameter simplenet0_conv1_weight (shape=(36, 0, 3, 3), dtype=)
  Parameter simplenet0_conv1_bias (shape=(36,), dtype=)
)

2. Nested block 測試:

我們來測試看看 nested block 的命名規則。

先定義一個 ScopedBlock1，它會把 scopedblock1_ 加入 children 的 key 中:

class ScopedBlock1(nn.HybridBlock):
    def __init__(self, **kargs):
        super(ScopedBlock1, self).__init__(**kargs)
        with self.name_scope():
            self.body = nn.HybridSequential()
            self.body.add(
                nn.Conv2D(20, 3),
                nn.Conv2D(36, 3),
            )

    def hybrid_forward(self, F, x, *args, **kwargs):
        return self.body(x)

再定義一個 ScopedBlock2，它會把 scopedblock2_ 加入 children 的 key 中:

class ScopedBlock2(nn.HybridBlock):

    def __init__(self, **kargs):
        super(ScopedBlock2, self).__init__(**kargs)
        with self.name_scope():
            self.body = nn.HybridSequential()
            for _ in range(4):
                self.body.add(ScopedBlock1())

    def hybrid_forward(self, F, x, *args, **kwargs):
        return self.body(x)

來看看 ScopedBlock2 的命名:

sb = ScopedBlock2()
sb.initialize()
print(sb.collect_params())

scopedblock20_ (
  Parameter scopedblock20_scopedblock10_conv0_weight (shape=(20, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock10_conv0_bias (shape=(20,), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock10_conv1_weight (shape=(36, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock10_conv1_bias (shape=(36,), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock11_conv0_weight (shape=(20, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock11_conv0_bias (shape=(20,), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock11_conv1_weight (shape=(36, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock11_conv1_bias (shape=(36,), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock12_conv0_weight (shape=(20, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock12_conv0_bias (shape=(20,), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock12_conv1_weight (shape=(36, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock12_conv1_bias (shape=(36,), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock13_conv0_weight (shape=(20, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock13_conv0_bias (shape=(20,), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock13_conv1_weight (shape=(36, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter scopedblock20_scopedblock13_conv1_bias (shape=(36,), dtype=<class 'numpy.float32'>)
)

答案其實也滿直觀的，就是命名會依序疊加所以變成類似 scopedblock20_scopedblock10_conv0_weight 這樣的名字。

3. Nested block 但不使用 name_scope:

那 Nested block 但不使用 name_scope 會發變怎樣？

class Block1(nn.HybridBlock):
    def __init__(self, **kargs):
        super(Block1, self).__init__(**kargs)
        self.body = nn.HybridSequential()
        self.body.add(
            nn.Conv2D(20, 3),
            nn.Conv2D(36, 3),
        )

    def hybrid_forward(self, F, x, *args, **kwargs):
        return self.body(x)

class Block2(nn.HybridBlock):

    def __init__(self, **kargs):
        super(Block2, self).__init__(**kargs)
        self.body = nn.HybridSequential()
        for _ in range(4):
            self.body.add(Block1())

    def hybrid_forward(self, F, x, *args, **kwargs):
        return self.body(x)

b = Block2()
b.initialize()
print(b.collect_params())

block20_ (
  Parameter conv0_weight (shape=(20, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter conv0_bias (shape=(20,), dtype=<class 'numpy.float32'>)
  Parameter conv1_weight (shape=(36, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter conv1_bias (shape=(36,), dtype=<class 'numpy.float32'>)
  Parameter conv2_weight (shape=(20, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter conv2_bias (shape=(20,), dtype=<class 'numpy.float32'>)
  Parameter conv3_weight (shape=(36, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter conv3_bias (shape=(36,), dtype=<class 'numpy.float32'>)
  Parameter conv4_weight (shape=(20, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter conv4_bias (shape=(20,), dtype=<class 'numpy.float32'>)
  Parameter conv5_weight (shape=(36, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter conv5_bias (shape=(36,), dtype=<class 'numpy.float32'>)
  Parameter conv6_weight (shape=(20, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter conv6_bias (shape=(20,), dtype=<class 'numpy.float32'>)
  Parameter conv7_weight (shape=(36, 0, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter conv7_bias (shape=(36,), dtype=<class 'numpy.float32'>)
)