Is a Picture Worth a Thousand Words? A Deep Learning Approach for Assessing the Impact of Investors’ Emotions Mined from Multimodal Social Postings on Corporate Credit Ratings


With the rise of the Social Web, it is increasingly more popular for investors to express their feelings about firms and products on online social media. Though previous studies have examined the relationship between investors’ sentiments captured in texts (e.g., online news articles) and corporate credit ratings, there is a research gap in terms of studying the relationship between investors’ emotions captured in multimodal social postings and corporate credit ratings. By drawing upon the appraisal theory and the notion of affect-as-information, this study aims to fill the current research gap by examining the influence of investors’ emotions (e.g., trust, joy, and anger) captured in multimodal social postings on corporate credit ratings. In particular, we apply state-of-the-art machine learning methods (e.g., topic modeling and deep convolutional neural networks) to extract investors’ emotions toward corporations from both texts and images posted to online social media. Our empirical results show that investors’ emotions such as “trust” and “joy” captured in textual social postings are positively associated with corporate credit rating, while investors’ emotion such as “anger” captured in the images of the social postings is negatively associated with corporate credit rating. Moreover, the negative influence of the “anger” emotion extracted from images is larger than the positive influence of “trust” and “joy” extracted from texts. The managerial implication of our study is that corporations should pay attention to both texts and images posted to online social media by investors as these postings affect corporate credit ratings, and subsequently the costs of corporate borrowings. Our findings also offer new insights for corporations, investors, and ratings agencies with respect to the predictive power of investors’ multimodal social posts toward corporate credit ratings.


Prof. Raymond Yiu Keung LAU
Department of Information Systems
City University of Hong Kong

Date & Time

11 Dec 2018 (Tuesday) 11:00 - 12:00


E11-4045 (University of Macau)

Organized by

Department of Computer and Information Science


RAYMOND Y.K. LAU is an Associate Professor in the Department of Information Systems at City University of Hong Kong. He has worked at the academia and the ICT industry for over twenty years. He is the author of more than 200 refereed international journals and conference papers. His research work is published in renowned journals such as IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), IEEE Transactions on Knowledge and Data Engineering, IEEE Intelligent Systems, IEEE Internet Computing, INFORMS Journal on Computing, MIS Quarterly, ACM Transactions on Information Systems, etc. His research interests include Financial Technology (FinTech), Social Media Analytics, Big Data Analytics, and Artificial Intelligence (AI) for Business. He is a senior member of the IEEE and the ACM, respectively.