KITHCT Chair Professor



Department of Computer Science

School of Computing

National University of Singapore



I am the KITHCT Chair Professor at the School of Computing, National University of Singapore. I was the Founding Dean of the School from 1998-2000. My main research interest is in multimedia information retrieval and social media analytics. In particular, my research focuses on the extraction, retrieval and question-answering of text, video and live media arising from the Web and social networks. I am also the Director of a joint research Center between NUS and Tsinghua (NExT) to research into big unstructured multi-source multimodal data analytics.


I am active in the international research community. I was the recipient of ACM SIGMM Technical Achievement Award 2015. I have also organized and served as program committee member of numerous international conferences in the areas of computer graphics, multimedia and text processing. I was the conference co-chair of ACM Multimedia 2005, ACM CIVR 2005, ACM SIGIR 2008 and ACM Web Science 2015. I serve in the editorial boards of: ACM Transactions of Information Systems (ACM), The Visual Computer (Springer Verlag), and Multimedia Tools and Applications (Kluwer). I am the chair of steering committee of ICMR (International Conference on Multimedia Retrieval) and Multimedia Modeling conference series.


I have also co-founded two technology startup companies based on research in NExT. I hold a PhD from the University of Leeds, UK.




My main research interest is in multimedia information retrieval, social media analysis and video processing. In particular, my research focuses on the extraction, retrieval and question-answering (QA) of text, video and live media arising from the Web and social networks. I am the Director of a multi-million-dollar joint Center between NUS and Tsinghua University in China to develop technologies for live media search. The project will gather, mine, search and organize user-generated contents within the cities of Beijing and Singapore. My group participates regularly in TREC-QA and TRECVID video retrieval evaluations and have achieved top positions; we have also won the A*STAR Grand Challenge on media search in 2008. More Research Details...


My scientific impact on Google Scholar

Recent Research Grants:

The list of grants that is active from 2008 onwards.

1)    Title: NExT++: Big Unstructured Data Analytics

       Collaborators: NUS, Tsinghua University, University of Southampton

       Source: NRF, Singapore

       Grant: S$12 million.

       Duration: Oct 2016 – Mar 2021

2)    Title: NExT: Extreme Search of User-Generated Contents

       Collaborators: NUS and Tsinghua University

       Source: NRF, Singapore

       Grant: S$11 million.

       Duration: May 2010 – Sep 2016

       Status: Completed

3)    Title: Intelligent Local Media Search

       Source: SPH, Singapore

       Approved fund: S$2.3 million

       Duration: Aug 2008 – Jun 2010

       Status: Completed

Supervision of graduate students

I have graduated 32 PhD Students since 2004

  • 2018: Aleksandr Farseev, Chen Jingyuan

  • 2017: Geng Xue, Fu Jie

  • 2016: Bian Jingwen, Song Xuemeng, Akbari Mohammad

  • 2014: Zhao Yiliang;  Zhang Hanwang;  Lin Jovian;  Ghasem Heyrani Nobari

  • 2013: Chen Xiangyu;  Amiri Hadi;  Yuan Jin;  Nie Liqiang

  • 2012: Li Guangda;  Katti Harish;  Yu Jianxing

  • 2011: Wang Kai;  Ming Zhaoyan

  • 2010: Zhang Yantao

  • 2009: Wang Gang

  • 2008: Xu Huaxin;  Shi Rui;  Maslennikov Mstislav;  Neo Shiyong

  • 2007: Marchenko Yelizaveta

  • 2006: Cui Hang

  • 2005: Xiao Jing

  • 2004: Zhao Yunlong;  Chaisorn Lekha;  Feng Huamin

I am currently supervising 10 PhD and Masters students as the main supervisor


  • Feng Fuli

  • Ivan Samborskii

  • Francesco Gelli

  • Kyaw Zaw Lin

  • Liao Lizi

  • Ma Yunshan

  • Shang Xindi

  • Wang Xiang

  • Zhao Na


  • Rahul Soni

Invited Talks


Invited presentations at scientific conferences/workshops

(since 2008 only)


  • Keynote Talk: “From Image to Video: the connection between Vision and language”. IEEE ICME, Hong Kong, Jul 2017


  • Keynote Talk: “Learning from Multiple Heterogeneous Social Networks”. ACM SIGIR Workshop on Heterogeneous Information Access, Pisa, Italy, Jul 2016.


  • Keynote Talk: “Multimedia Retrieval: Are We Doing it Right?” ACM ICMR (Int’l Conference on Multimedia Retrieval), Shanghai, China, Jun 2015.


  • Keynote Talk: “From Social Media Data to Actionable Analytics”. Workshop on SoMeRA’14 (International Workshop on Social Media Retrieval and Analysis), ACM SIGIR ‘2014, Australia, Jul 2014.


  • Keynote Speaker: “Managing large-scale social images as part of a live Social Media Observatory”. The 13th Pacific-Rim Conference on Multimedia (PCM2012), Singapore, Dec 2012.

  • Keynote Speaker: “Live Social Monitors: Learning the Social Pulses of a City from User-Generated Contents”. The 13th Int’l Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Dublin, Ireland, May 2012.


  • Keynote Speaker: “Learning the Social Pulses of a City from User-Generated Contents”, Workshop on Social Behavioural Networked Media Access (SBNMA’11), in ACM Multimedia ‘2011, Arizona, USA, 28 Nov – 1 Dec 2011.

  • Keynote Speaker: “Extracting Structured Knowledge from User-Generated Information Resources”, Workshop on SWSM (Social Web Search and Mining, Analysis under Crisis), in ACM SIGIR ‘2011, Beijing, China, 24-28 July 2011.

  • Keynote Speaker: “Multimedia Question-Answering (MMQA)”, MMM ‘2011 (International Conference on Multimedia Modeling), Taipei, Taiwan, Jan 2011.


  • Keynote Speaker: “Extracting Knowledge form Community QA (cQA) Sites”, COLING Workshop (People’s Web Meets NLP), COLING’10 (23rdInternational Conference on Computational Linguistics 2010), Beijing, China, 23-28 August 2010.

  • Keynote Speaker: “Towards Web-Scale Media Content Analysis and Retrieval”, CIVR’2010 (International Conference on Image and Video Retrieval 2010), Xi’an, China, 5-7 July 2010.


  • Keynote Speaker: “Web-Scale Media Search: From Research Prototype to Commercial Applications”,  DICTA’2009 (Digital Image Computing: Techniques and Applications), Melbourne, 1-3 December 2009.

  • Keynote Speaker: “Visual Dictionary: Toward a Higher-level Visual Representation for Object Categorization”,  SIGFRA’09 (The Singaporean-French IPAL Symposium), Singapore, 18-20 February, 2009


  • Keynote Speaker: “A Unified MMM Model for Media Content Analysis and Retrieval”, MMM 2008 (International Conference on Multimedia Modeling). Kyoto, Japan, January 2008

Career History:

  • 2009 – present: KITHC Chair Professor, School of Computing, NUS

  • 2003 – present: Professor, School of Computing, NUS

  • 1998-2000: Acting Dean, School of Computing, NUS

  • 1996-1998: Acting Head, Department of Information Systems & Computer Science, Faculty of Science, NUS

  • 1996-2003: Associate Professor, School of Computing, NUS

  • 1990-1996: Senior Lecturer, Dept of Info Systems & Computer Science, NUS

  • 1988-1990: Research Staff member, Institute of Systems Science (now I2R)

  • 1983-1988: Lecturer, Dept of Information Systems & Computer Science, NUS



  • ACM SIGMM Technical Achievement Award 2015 (for Outstanding Technical Contributions to Multimedia Computing, Communications and Applications)

Visiting Professorship:

  • 2017: Distinguished Visiting Professor (杰出访问教授), Tsinghua University, China.

Associate Editors of Journals

  • IEEE Transactions of Multimedia (IEEE) – (2000-2004)

  • The Visual Computer (Springer-Verlag) – (1993-present)

  • Multimedia Tools and Applications (Kluwer) – (1998 – present)

  • ACM Transaction of Information Systems (ACM) – (2007 – present)

  • Foundations and Trends in Information Retrieval (FnTIR) – (2010 – 2017)


Steering Committee Members

  • Multimedia Modeling Conference (since 1993)

  • ACM International Conference on Multimedia Retrieval (ICMR) (as Member since 2008, and as Chair since 2015)

Conference General Co-Chairs

  • ACM Web Science – Jun 2015 (Oxford, UK)

  • ACM SIGIR – Aug 2008 (Singapore)

  • ACM Multimedia - Nov 2005 (Singapore)

  • CIVR (Conference on Image & Video Retrieval) – Jul 2005 (Singapore)

  • Computer Graphics International (CGI)- 1990 (Singapore); 2000 (Switzerland); 2001 (Hong Kong); 2003 (Japan)

  • Multi-Media Modeling (MMM)-- 1993 (Singapore); 1995 (Singapore); 1997 (Singapore); 2003 (Taiwan)

  • Pacific Graphic-- 1998 (Singapore)


Conference Technical Program Co-Chairs

  • ACM ICMR Co-Chair – Apr 2013 (USA)

  • ACM SIGIR PC Co-Chair – Aug 2011 (China)

  • CIVR (Conference on Image & Video Retrieval) – Jul 2009 (Greece)

  • Numerous others

Technical Program Committees and Area Chairs

  • Numerous conferences in computer graphics and multimedia areas, including: ACM Multimedia, ACM SIGIR, ACL (Association of Computational Linguistics), EMNLP, WWW, Computer Graphics International (CGI), Computer Animation, Pacific Graphics, Multimedia Modeling (MMM), IEEE Int’l Conference on MM and Expo (ICME)

Member of Visiting Committee

  • Member of Review Panel: Program for “Interactive Multimodal Information Management”, under Swiss National “Centres of Competence in Research" (NCCR IM2), Switzerland (2006-2013)

  • Member of Industrial Advisory Board: European Network of Excellence Program – INTERMEDIA ( Europe (2007-2010)

Services in Singapore

Professional Services




CS 3246:

Multimedia Content Analysis and Search

Aims and Objectives:

This course introduces the concepts, issues, design, implementation, standards and applications of multimedia technologies, with special emphasis on media content analysis and search. The media to be considered include text, digital image, digital video and their integration. At the end of this course, the students should have the expertise and competence to design, implement and deploy multimedia search systems, with a good understanding of the architecture of modern search engines.


Brief Description:

The term “multimedia” essentially refers to the “means of communications through multiple media”. Although human-to-human communications are essentially multi-sensory and multi-modal, until recently, human-computer communications have been largely text-based. The rapid development in last 20 years has enable technologies for multi-modal communications and analysis to become a reality. Also, the emergence of Web 2.0 and social network has resulted in the wide availability of and demand for multimedia contents. The ability to analyze, index and retrieve such multimedia contents, especially as they are being produced in real-time, will be of paramount importance.

This module presents techniques for the representation, compression, storage, analysis and retrieval of Web-scale digital multimedia information, and their implementation as media search engines. The module is divided into 2 parts. The first part discusses the characteristics, mathematical foundation, compression, processing and applications of digital multimedia data including: text, image and video. It also covers standards in digital multimedia data such as JPEG, MPEG & H261, etc. The second part covers the analysis and retrieval of multimedia contents, including text, image and video. The course will also introduce modern architecture for text and image search engines.



1.   Introduction to multimedia systems: basic concepts in multimedia; applications in media search & social media analysis; summary

2.   Fundamentals of digital images and transformations: digital image representation; color look-up tables; dithering and filtering; Instagram-style applications.

3.   Multimedia Compression Techniques: Principles of compression; basic digital compression techniques; Hoffman & Arithmetic Coding; LZW; Predictive Coding

4.   JPEG compression standards: lossy compression schemes; device independent color models; transformed based compression techniques; GIF and JPEG.

5.   Text representation and retrieval: free text representation; term-based feature extraction and weighting; vector-space retrieval model; relevance feedback; indexing schemes; architecture of a text search engine; Lab exercise on text search

6.   Image content analysis and retrieval: visual features, feature extractions; similarity metrics; indexing (inverted index and hash-based index) and retrieval; architecture and UI of image retrieval systems; Lab practices on image search

7.   Digital video: digital video fundamentals; video compression principles and techniques; MPEG standards: MPEG-1 & MPEG-2; introduction to MPEG-4; H261 video conferencing standards.

8.   Future of media search: mobile search, social media search and vertical domain search.

CS 4242:

Social Media Computing

Aims and Objectives:

This module introduces the background and present states of social networks and their analysis in terms of contents, users, social relations and applications. The social network to be covered include microblogs sites like Twitter, social communication sites like Facebook, location sharing sites like 4Square, and photo sharing sites like Instagram and Flicker. At the end of this module, students are expected to have good understanding of the background, design, analysis and implementation of social media analysis systems.


Brief Description:

The emergence of WWW, smart mobile devices and social networks has revolutionized the way we communicate, create, disseminate, and consume information. This has ushered in a new era of communications that involves complex information exchanges and user relationships. This module aims to study the social network phenomena by analyzing the complex social relation networks between users, the contents they shared, and the ways contents and events are perceived and propagated through the social networks. The analysis will provide better understanding of the concerns and interests of users, and help to uncover the live and emerging events that will affect the community.


  1. Introduction to social networks: types of social networks, the targeted users, social impacts, privacy and trust issues

  2. Overview of social media analysis framework; information gathering, storage and analysis; applications; truth and reliability

  3. The users and relationship networks: graph theory and social networks; information networks and WWW; analysis of network dynamics based on population and structural models; markets and strategic interactions in networks; applications

  4. The social media contents: text, social text, images, videos, location check-ins; their crawling, feature extraction, analysis, fusion, indexing and search

  5. Analytics: location, people and organization analytics; trend analysis, user communities, detection of live and emerging events

  6. Applications: social assistance and recommendations

  7. Future trends



A full list of my publications can be found on my dblp page here.

List of Best Paper Awards:


  • 2016 Honorable Mention Award., ACM SIGIR’ 2016
    (Zhang et al., Discrete Collaborative Filtering, Pisa Italy, Jul 2016)


  • 2015 IEEE Multimedia Best Paper Award, IEEE Multimedia Magazine
    (Hu et al., Towards Multi-Screen Social TV with Geolocation-Aware Social Sense, 31(3): 10-19, 2014)

  • 2015 Prize Paper Award Honorable Mention, IEEE Transactions on Multimedia
    (Hong et al., Movie2Comics: Towards a Lively Video Content Presentation, 14(3-2): 858-870, 2012)

  • 2015 Best Paper Runner-up Award, MMM ‘2015
    (Gao et al. Multimedia Social Event Detection in Microblog, Sydney, Australia, Jan 2015)


  • 2013 Best Student Paper in ACM Multimedia
    (Zhang et al., Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval. 33-42, Barcelona, Spain, Oct 2013).


  • 2011 Best Paper Award, MMM‘2011
    (Zhao et al. Tat-Seng Chua. Generating Representative Views of Landmarks via Scenic Theme Detection, Taiwan, Jan 2011).


  • 2010 Best Paper Award, ACM Multimedia 2010
    (Hong et al., Dynamic Captioning: Video Accessibility Enhancement for Hearing Impairment, Florence, Italy, Oct 2010)


  • 2008 Best Student Paper Award, MMM’ 2008
    (Zheng et al., Object-based Image Retrieval beyond Visual Appearances, Tokyo, Japan. Jan 2008)


  • 2007 Best Paper Award, MMM’ 2007
    (Marchenko et al., Ontology-based Annotation of Paintings using Transductive Inference Framework, Springer-Verlag, LNCS 4351. 13-23.

  • 2007 Best Paper Award, PCM ‘2007
    (Marchenko et al. Auto-Annotation of Paintings using Social Annotations, Domain Ontology and Transductive Inference, Hong Kong, Dec 2007)