Domain Randomization for Neural Network Classification
Svetozar Zarko Valtchev
Jianhong Wu
York University, Toronto
[Paper]
[Dataset]
[Talk]
[Slides]
[Bibtex]
First 4 real and synthetic images in each category, and their subsequent analysis as to what each model is looking at to classify each image (real model on the left, synthetic on the right). Second row shows the GradCAM explainer while the third row utilizes Occulusion Sensitivity (OS), when testing for the cat class.


Abstract

Large data requirements are often the main hurdle in training neural networks. Synthetic data is a cheap and efficient solution to assemble such large datasets. Using domain randomization, we show that a sufficiently well generated synthetic image dataset can be used to train a neural network classifier, achieving accuracy levels as high as 88% on 2 category classification. We show that the most important domain randomization parameter is a large variety of subjects, while secondary parameters such as lighting and textures are not. Based on our results, there is reason to believe that models trained on domain randomized images transfer to new domains better than those trained on real photos. Model performance seems to diminish slightly as the number of categories increases.

Talk


[Slides]

Dataset

We generate 25,000 synthetic images with labels of cats, dogs, cars and bicycles. The dataset is publically avaiable on Kaggle. Please cite the publication if you choose to use the dataset.

[Dataset]

Results

We found that the most important parameter in the accuracy of the neural-based classifier was the variety of breeds of the subjects. All other parameters had marginal effects on the overall accuracy, with the occluders and the variation of poses actually having negative consequences. This is in line with what we expect, obstructing parts of the subject should cause important features to be hidden, as is the case with the obstructors and some specific poses. Results remained consistent as we increased the number of categories in the classification task, adding car and bike images to the experiments.

Paper and Supplementary Material

S.Z. Valtchev, J. Wu.
Domain Randomization for Neural Network Classification.
SpringerOpen Journal of Big Data, 2021.

[Bibtex]


Acknowledgements

This work has been supported by the Natural Sciences and Engineering Research Council of Canada, and by the Canada Research Chairs program.