 # CNN in numpy

## Let's calculate CNN manually

Posted by Chris Hoyean Song on November 26, 2017

# CNN in numpy

The purpose of this article is to understand internal calculations of `CNN(Convolution Neural Network)`.

Most of ML applications are actively using CNN(Convolution Neural Network). It is quite easy to create a CNN layer thanks to `Google Tensorflow`. But, understanding its internal logic from scratch will help you to develop and improve your models.

I found a great illustration of CNN in `Stanford CS231n lecture`. So, I decided to use these inputs (7x7x3) and filter weights (3x3x3) as the initial values. Also, I’ll derive the 1-step CNN result using Tensorflow and numpy. Credit for this image goes to Andrej Karpathy.

# 1. Setting up the initial values

First, I’ll set the initial values for CNN. We need input values `x` and weights `w`. The initial values of input values and weights are from the above image of `Stanford CS231n lecture`.

``````
import numpy as np
import tensorflow as tf
from tensorflow.python.framework import dtypes

# 1. Setting up initial values

x = np.zeros((7, 7, 3))
x[:, :, 0] = np.mat(
"0 0 0 0 0 0 0;0 0 1 0 1 0 0;0 2 1 0 1 2 0;0 0 2 0 0 1 0;0 2 0 1 0 0 0;0 0 0 1 2 2 0;0 0 0 0 0 0 0"
).A
x[:, :, 1] = np.mat(
"0 0 0 0 0 0 0;0 1 0 0 1 1 0;0 0 2 2 1 1 0;0 2 1 2 1 0 0;0 2 1 1 2 2 0;0 1 2 0 2 2 0;0 0 0 0 0 0 0"
).A
x[:, :, 2] = np.mat(
"0 0 0 0 0 0 0;0 2 1 1 1 1 0;0 2 2 1 2 1 0;0 1 1 0 2 2 0;0 2 1 2 2 0 0;0 1 2 2 0 0 0;0 0 0 0 0 0 0"
).A

x = np.reshape(x, (1, 7, 7, 3))
# print("x:",x)

w = np.zeros((3, 3, 3, 2))
w[:, :, 0, 0] = np.mat("0 0 1;-1 1 1;0 1 0").A
w[:, :, 1, 0] = np.mat("1 1 1;0 1 1;0 1 0").A
w[:, :, 2, 0] = np.mat("-1 0 0;-1 1 1;0 -1 0").A

# w1 = np.zeros((3,3,3))
w[:, :, 0, 1] = np.mat("0 0 0;1 1 -1;-1 1 1").A
w[:, :, 1, 1] = np.mat("0 1 -1;1 1 -1;-1 1 -1").A
w[:, :, 2, 1] = np.mat("1 1 0;-1 -1 0;0 -1 1").A

stride = 2
scope = "conv_in_numpy"
act = tf.nn.relu  # activation
nf = 2  # number of filters
rf = 3  # filter size
b = [1, 0]  # bias

np_o = np.zeros((1, 3, 3, 2))
s = stride

``````

# 2. Convolution in Tensorflow

On this step, I’ll get the 1-step CNN result using Tensorflow.

``````
# 2. CNN in Tensorflow

print("--- Convolution in Tensorflow ---")

tf_x = tf.constant(x, dtype=dtypes.float32)

with tf.Session() as sess:
with tf.variable_scope(scope):
nin = tf_x.get_shape().value
tf_w = tf.get_variable("w", [rf, rf, nin, nf], initializer=tf.constant_initializer(w))
tf_b = tf.get_variable(
"b", [nf],
initializer=tf.constant_initializer(b, dtype=dtypes.float32))
tf_z = tf.nn.conv2d(
tf_h = act(tf_z)
sess.run(tf.global_variables_initializer())
tf_o = sess.run(tf_z)
print("tf_o0:\n", tf_o[0, :, :, 0])
print("tf_o1:\n", tf_o[0, :, :, 1])
``````
```--- Convolution in Tensorflow ---
tf_o0:
[[  6.   4.   3.]
[ 13.   7.   4.]
[ 10.   9.   5.]]
tf_o1:
[[ -1.  -3.   1.]
[  0.   6.   2.]
[  1.  -3.  12.]]
```

# 3. Convolution in numpy

Okay, it is time to calculate CNN manually. And its code is impressively simple.

``````
# 3. CNN in numpy

print("--- Convolution in numpy ---")

for z in range(nf):
# print("z:", z)
h_range = int((x.shape - rf) / s) + 1  # (W - F + 2P) / S
for _h in range(h_range):
w_range = int((x.shape - rf) / s) + 1  # (W - F + 2P) / S
for _w in range(w_range):
np_o[0, _h, _w, z] = np.sum(
x[0, _h * s:_h * s + rf, _w * s:_w * s + rf, :] *
w[:, :, :, z]) + b[z]

print("np_o0:\n", np_o[0, :, :, 0])
print("np_o1:\n", np_o[0, :, :, 1])
``````
```--- Convolution in numpy ---
np_o0:
[[  6.   4.   3.]
[ 13.   7.   4.]
[ 10.   9.   5.]]
np_o1:
[[ -1.  -3.   1.]
[  0.   6.   2.]
[  1.  -3.  12.]]
```

Finally, we can check the results are same each other using the np.testing library.

``````np.testing.assert_almost_equal(tf_o, np_o)
``````

# 5. Full Source Code

Here is the full source code on my gist.