Image Captioning Based on Deep Neural Network...

Image Captioning Based on Deep Neural Networks Transforming Objects into Words

Author Name : Mr. Rahul Shivhare, Mr. Saurabh Bhardwaj, Neha Paliwal, Pooja Chaudhary, Bhawana Goel

ABSTRACT

In recent years, with the development of deep learning, the combination of computer vision and natural language processing has aroused great attention in the past few years. Image captioning is a representative of this field, which makes the computer learn to use one or more sentences to understand the visual content of an image. The meaningful description generation process of high level image semantics requires not only the recognition of the object and the scene, but the ability of analyzing the state, the attributes and the relationship among these objects. In this paper, we mainly describe three image captioning methods using the deep neural networks: CNN-RNN based, CNN-CNN based and Reinforcement-based framework. Then we introduce the representative work of these three top methods respectively. Furthermore, the advantages and the shortcomings of these methods are discussed, providing the commonly used datasets and evaluation criteria in this field. Finally, this paper highlights some open challenges in the image caption task.