Traffic Sign Classification with Convolutional Neural Networks in Python

Implementing Traffic Sign Classification in Python for Computer Vision tasks in Autonomous Vehicles.

Test online here

Content

Short description of the content. Full codes you can find inside the course by link above:

• Loading Data
  • Unique Training Examples
  • Histogram of Training Examples
  • Good Quality Examples
  • Table of Labels
• Preprocessing Data
• Model 1
  • Overfitting Small Data for Model 1
  • Training of Model 1
  • Visualizing Filters for Model 1
  • Visualizing Feature Maps for Model 1
• Predicting with image from test dataset
• Predicting with user’s image
• Traffic Sign Classification in Real Time

Loading Data

Data used for this task is German Traffic Sign Benchmarks (GTSRB).
Initially datasets consist of images in PPM format with different sizes.

For current task datasets were organized as it was done for CIFAR-10 Image Classification:

• x_train, x_validation, x_test - 4D tensors as numpy.ndarray type with shapes (12345, 3, 32, 32)
• y_train, y_validation, y_test - 1D tensors as numpy.ndarray type with shapes (12345, )

Here,
12345 - number of images / labels,
3, 32, 32 - image with 3 channels and size of 32x32 (height and width).

All tensors were put in a dictionary and were written in a pickle file:

d = {'x_train': x_train, 'y_train': y_train,
     'x_validation': x_validation, 'y_validation': y_validation,
     'x_test': x_test, 'y_test': y_test}

Unique Training Examples

Examples of Unique Traffic Signs for every class from training dataset are shown on the figure below.

Histogram of Training Examples

Histogram of 43 classes for training dataset with their number of examples for Traffic Signs Classification before and after equalization by adding transformated images from original dataset is shown on the figure below. After equalization, training dataset has increased up to 86989 examples.

Good Quality Examples

Examples of Good Quality Traffic Signs for every class to show in GUI for driver are shown on the figure below.

Table of Labels

Following tabel represents number of class and its corresponding label (description).

Class	Description
0	Speed limit (20km/h)
1	Speed limit (30km/h)
2	Speed limit (50km/h)
3	Speed limit (60km/h)
4	Speed limit (70km/h)
5	Speed limit (80km/h)
6	End of speed limit (80km/h)
7	Speed limit (100km/h)
8	Speed limit (120km/h)
9	No passing
10	No passing for vehicles over 3.5 metric tons
11	Right-of-way at the next intersection
12	Priority road
13	Yield
14	Stop
15	No vehicles
16	Vehicles over 3.5 metric tons prohibited
17	No entry
18	General caution
19	Dangerous curve to the left
20	Dangerous curve to the right
21	Double curve
22	Bumpy road
23	Slippery road
24	Road narrows on the right
25	Road work
26	Traffic signals
27	Pedestrians
28	Children crossing
29	Bicycles crossing
30	Beware of ice/snow
31	Wild animals crossing
32	End of all speed and passing limits
33	Turn right ahead
34	Turn left ahead
35	Ahead only
36	Go straight or right
37	Go straight or left
38	Keep right
39	Keep left
40	Roundabout mandatory
41	End of no passing
42	End of no passing by vehicles over 3.5 metric tons

Preprocessing Data

Prepared data is preprocessed in variety of ways and appropriate datasets are written into ‘pickle’ files.
Datasets data0 - data3 have RGB images and datasets data4 - data8 have Gray images.

• data0.pickle - Shuffling
• data1.pickle - Shuffling, /255.0 Normalization
• data2.pickle - Shuffling, /255.0 + Mean Normalization
• data3.pickle - Shuffling, /255.0 + Mean + STD Normalization
• data4.pickle - Grayscale, Shuffling
• data5.pickle - Grayscale, Shuffling, Local Histogram Equalization
• data6.pickle - Grayscale, Shuffling, Local Histogram Equalization, /255.0 Normalization
• data7.pickle - Grayscale, Shuffling, Local Histogram Equalization, /255.0 + Mean Normalization
• data8.pickle - Grayscale, Shuffling, Local Histogram Equalization, /255.0 + Mean + STD Normalization

Shapes of data0 - data3 are as following (RGB):

• x_train: (86989, 3, 32, 32)
• y_train: (86989,)
• x_validation: (4410, 3, 32, 32)
• y_validation: (4410,)
• x_test: (12630, 3, 32, 32)
• y_test: (12630,)

Shapes of data4 - data8 are as following (Gray):

• x_train: (86989, 1, 32, 32)
• y_train: (86989,)
• x_validation: (4410, 1, 32, 32)
• y_validation: (4410,)
• x_test: (12630, 1, 32, 32)
• y_test: (12630,)

Examples of some of them (RGB, Gray, Local Histogram Equalization) are shown on the figure below:

Model 1

For Model 1 architecture will be used as it was done for CIFAR-10 Image Classification:
Input –> Conv –> ReLU –> Pool –> Affine –> ReLU –> Affine –> Softmax

For Model 1 following parameters will be used:

Parameter	Description
Weights Initialization	HE Normal
Weights Update Policy	Adam
Activation Functions	ReLU
Regularization	L2
Pooling	Max
Loss Functions	Softmax

Overfitting Small Data for Model 1

For Overfitting Small Data of Model 1 dataset ‘data8.pickle’ was chosen.

Overfitting Small Data with 10 training examples and 100 epochs is shown on the figure below.

Training of Model 1

For training Model 1 dataset ‘data8.pickle’ was chosen as it reached the best accuracy over all datasets.

Model 1 with ‘data8.pickle’ dataset reached 0.989 training accuracy.
Training process of Model 1 with 17 500 iterations is shown on the figure below.

Visualizing Filters

Initialized and Trained filters for CNN Layer are shown on the figure below.

Visualizing Feature Maps

Feature maps of trained network for CNN Layer are shown on the figure below.

Predicting with image from test dataset

Prediction with image from test dataset ‘data8.pickle’ is shown on the figure below.
Classified as Speed limit (60km/h).

Predicting with user’s image

Prediction with user’s image is shown on the figure below.
Classified as Keep right.

Traffic Sign Classification in Real Time

Traffic Sign Classification with Convolutional Neural Network. Left: original frame with detected Sign. Upper Right: cut frame with detected Sign. Lower Right: classified frame by ConvNet according to the detected Sign.

MIT License

Copyright (c) 2019 Valentyn N Sichkar

github.com/sichkar-valentyn

Reference to:

Valentyn N Sichkar. Neural Networks for computer vision in autonomous vehicles and robotics // GitHub platform. DOI: 10.5281/zenodo.1317904