publications | Gaurav Sahu

2025

Future Sight: Fine-Tuning Language Models for Dynamic Story Generation

Brian Zimmerman, Gaurav Sahu, and Olga Vechtomova

In International Conference on Computational Intelligence in Music, Sound, Art and Design (Part of EvoStar), 2025

2024

Harnessing Generalist LLMs for Diverse Objective and Subjective NLP Tasks

Gaurav Sahu

2024
Insightbench: Evaluating business analytics agents through multi-step insight generation

Gaurav Sahu, Abhay Puri, Juan Rodriguez, and 8 more authors

arXiv preprint arXiv:2407.06423, 2024
Diversity in issue assignment: humans vs bots

Aniruddhan Murali, Gaurav Sahu, Kishanthan Thangarajah, and 3 more authors

Empirical Software Engineering, 2024
Online Seminar-Style Presentation-Based Graduate Courses in STEM: Reflection and Recommendations for Tomorrow’s Professionals

R Cohen, E Awad, G Sahu, and 1 more author

Creative Education, 2024
LitLLMs, LLMs for Literature Review: Are we there yet?

Shubham Agarwal^*, Gaurav Sahu^*, Abhay Puri^*, and 5 more authors

arXiv preprint arXiv:2412.15249, 2024
Computational Modeling of Artistic Inspiration: A Framework for Predicting Aesthetic Preferences in Lyrical Lines Using Linguistic and Stylistic Features

Gaurav Sahu and Olga Vechtomova

arXiv preprint arXiv:2410.02881, 2024

2023

Llm aided semi-supervision for efficient extractive dialog summarization

Nishant Mishra, Gaurav Sahu, Iacer Calixto, and 2 more authors

In Findings of the Association for Computational Linguistics: EMNLP 2023, 2023
Promptmix: A class boundary augmentation method for large language model distillation

Gaurav Sahu, Olga Vechtomova, Dzmitry Bahdanau, and 1 more author

arXiv preprint arXiv:2310.14192, 2023
Online Delivery of Discussion-Oriented Postsecondary Classes during Critical Health Scenarios, Educating Responsible Computer Scientists towards Improved Societal Trust in Artificial Intelligence

Robin Cohen, Anna Xing, Hongyi Chen, and 2 more authors

Creative Education, 2023

2022

arXiv

LyricJam Sonic: A Generative System for Real-Time Composition and Musical Improvisation

Olga Vechtomova and Gaurav Sahu

Oct 2022

Abs DOI

Electronic music artists and sound designers have unique workflow practices that necessitate specialized approaches for developing music information retrieval and creativity support tools. Furthermore, electronic music instruments, such as modular synthesizers, have near-infinite possibilities for sound creation and can be combined to create unique and complex audio paths. The process of discovering interesting sounds is often serendipitous and impossible to replicate. For this reason, many musicians in electronic genres record audio output at all times while they work in the studio. Subsequently, it is difficult for artists to rediscover audio segments that might be suitable for use in their compositions from thousands of hours of recordings. In this paper, we describe LyricJam Sonic – a novel creative tool for musicians to rediscover their previous recordings, re-contextualize them with other recordings, and create original live music compositions in real-time. A bi-modal AI-driven approach uses generated lyric lines to find matching audio clips from the artist’s past studio recordings, and uses them to generate new lyric lines, which in turn are used to find other clips, thus creating a continuous and evolving stream of music and lyrics. The intent is to keep the artists in a state of creative flow conducive to music creation rather than taking them into an analytical/critical state of deliberately searching for past audio segments. The system can run in either a fully autonomous mode without user input, or in a live performance mode, where the artist plays live music, while the system ’listens’ and creates a continuous stream of music and lyrics in response.
NLP4ConvAI

Data Augmentation for Intent Classification with Off-the-shelf Large Language Models

Gaurav Sahu, Pau Rodriguez, Issam Laradji, and 3 more authors

In Proceedings of the 4th Workshop on NLP for Conversational AI, May 2022

Abs DOI

Data augmentation is a widely employed technique to alleviate the problem of data scarcity. In this work, we propose a prompting-based approach to generate labelled training data for intent classification with off-the-shelf language models (LMs) such as GPT-3. An advantage of this method is that no task-specific LM-fine-tuning for data generation is required; hence the method requires no hyper parameter tuning and is applicable even when the available training data is very scarce. We evaluate the proposed method in a few-shot setting on four diverse intent classification tasks. We find that GPT-generated data significantly boosts the performance of intent classifiers when intents in consideration are sufficiently distinct from each other. In tasks with semantically close intents, we observe that the generated data is less helpful. Our analysis shows that this is because GPT often generates utterances that belong to a closely-related intent instead of the desired one. We present preliminary evidence that a prompting-based GPT classifier could be helpful in filtering the generated data to enhance its quality.
JDSA

Personalized multi-faceted trust modeling to determine trust links in social media and its potential for misinformation management

Alexandre Parmentier, Robin Cohen, Xueguang Ma, and 2 more authors

In International Journal of Data Science and Analytics, Jan 2022

Abs DOI

In this paper, we present an approach for predicting trust links between peers in social media, one that is grounded in the artificial intelligence area of multiagent trust modeling. In particular, we propose a data-driven multi-faceted trust modeling which incorporates many distinct features for a comprehensive analysis. We focus on demonstrating how clustering of similar users enables a critical new functionality: supporting more personalized, and thus more accurate predictions for users. Illustrated in a trust-aware item recommendation task, we evaluate the proposed framework in the context of a large Yelp dataset. We then discuss how improving the detection of trusted relationships in social media can assist in supporting online users in their battle against the spread of misinformation and rumours, within a social networking environment which has recently exploded in popularity. We conclude with a reflection on a particularly vulnerable user base, older adults, in order to illustrate the value of reasoning about groups of users, looking to some future directions for integrating known preferences with insights gained through data analysis.

2021

ICCC

LyricJam: A system for generating lyrics for live instrumental music

Olga Vechtomova, Gaurav Sahu, and Dhruv Kumar

In Proceedings of ICCC, Sep 2021

Abs

We describe a real-time system that receives a live audio stream from a jam session and generates lyric lines that are congruent with the live music being played. Two novel approaches are proposed to align the learned latent spaces of audio and text representations that allow the system to generate novel lyric lines matching live instrumental music. One approach is based on adversarial alignment of latent representations of audio and lyrics, while the other approach learns to transfer the topology from the music latent space to the lyric latent space. A user study with music artists using the system showed that the system was useful not only in lyric composition, but also encouraged the artists to improvise and find new musical expressions. Another user study demonstrated that users preferred the lines generated using the proposed methods to the lines generated by a baseline model.
AAMAS

Towards A Multi-agent System for Online Hate Speech Detection

Gaurav Sahu, Robin Cohen, and Olga Vechtomova

In Proceedings of the 2nd International Workshop on Autonomous Agents for Social Good (AASG), AAMAS, May 2021

Abs Poster

This paper envisions a multi-agent system for detecting the presence of hate speech in online social media platforms such as Twitter and Facebook. We introduce a novel framework employing deep learning techniques to coordinate the channels of textual and image processing. Our experimental results aim to demonstrate the effectiveness of our methods for classifying online content, training the proposed neural network model to effectively detect hateful instances in the input. We conclude with a discussion of how our system may be of use to provide recommendations to users who are managing online social networks, showcasing the immense potential of intelligent multi-agent systems towards delivering social good.
EACL

Adaptive Fusion Techniques for Multimodal Data

Gaurav Sahu and Olga Vechtomova

In Proceedings of EACL, Apr 2021

Abs Poster

Effective fusion of data from multiple modalities, such as video, speech, and text, is challenging due to the heterogeneous nature of multimodal data. In this paper, we propose adaptive fusion techniques that aim to model context from different modalities effectively. Instead of defining a deterministic fusion operation, such as concatenation, for the network, we let the network decide “how” to combine a given set of multimodal features more effectively. We propose two networks: 1) Auto-Fusion, which learns to compress information from different modalities while preserving the context, and 2) GAN-Fusion, which regularizes the learned latent space given context from complementing modalities. A quantitative evaluation on the tasks of multimodal machine translation and emotion recognition suggests that our lightweight, adaptive networks can better model context from other modalities than existing methods, many of which employ massive transformer-based networks.

2020

COLING

Adversarial Learning on the Latent Space for Diverse Dialog Generation

Kashif* Khan, Gaurav* Sahu, Vikash* Balasubramanian, and 2 more authors

In Proceedings of COLING, Dec 2020

Abs DOI Poster

Generating relevant responses in a dialog is challenging, and requires not only proper modeling of context in the conversation, but also being able to generate fluent sentences during inference. In this paper, we propose a two-step framework based on generative adversarial nets for generating conditioned responses. Our model first learns a meaningful representation of sentences by autoencoding, and then learns to map an input query to the response representation, which is in turn decoded as a response sentence. Both quantitative and qualitative evaluations show that our model generates more fluent, relevant, and diverse responses than existing state-of-the-art methods.
ISMIR

Generation of lyrics lines conditioned on music audio clips

Olga Vechtomova, Gaurav Sahu, and Dhruv Kumar

In Proceedings of the First Workshop on NLP for Music and Audio (NLP4MusA) at ISMIR, "16 " # "oct" 2020

Abs

We present a system for generating novel lyrics lines conditioned on music audio. A bimodal neural network model learns to generate lines conditioned on any given short audio clip. The model consists of a spectrogram variational autoencoder (VAE) and a text VAE. Both automatic and human evaluations demonstrate effectiveness of our model in generating lines that have an emotional impact matching a given audio clip. The system is intended to serve as a creativity tool for songwriters.
UWSpace

Adaptive Fusion Techniques for Effective Multimodal Deep Learning

Gaurav Sahu

University of Waterloo, "16 " # "oct" 2020

Abs

Effective fusion of data from multiple modalities, such as video, speech, and text, is a challenging task due to the heterogeneous nature of multimodal data. In this work, we propose fusion techniques that aim to model context from different modalities effectively. Instead of defining a deterministic fusion operation, such as concatenation, for the network, we let the network decide “how” to combine given multimodal features more effectively. We propose two networks: 1) Auto-Fusion network, which aims to compress information from different modalities while preserving the context, and 2) GAN-Fusion, which regularizes the learned latent space given context from complementing modalities. A quantitative evaluation on the tasks of multimodal machine translation and emotion recognition suggests that our adaptive networks can better model context from other modalities than all existing methods, many of which employ massive transformer-based networks.
CE Journal

Digital Literacy for Secondary School Students: Using Computer Technology to Educate about Credibility of Content Online

Robin Cohen, Alexandre Parmentier, Glaucia Melo, and 8 more authors

In Creative Education Journal special issue on Education and Information Technology, "16 " # "oct" 2020

Abs

This paper presents an approach to educate secondary school students in the province of Ontario about the credibility of online content. The critical focus here is on integrating computer technology into the teaching of the topic; how to introduce the material in classroom settings with respect to the current curriculum is also outlined. Contrast with an existing proposal for digital literacy developed by historians at Stanford University is provided at the outset. In all, the value of appealing to the current digital experiences of students, when revealing the potential for misinformation, is the critical message. Exploration of social media environments popular with youth and opportunities for game-based quizzes for interactive engagement are both advocated.

2019

arXiv

Multimodal speech emotion recognition and ambiguity resolution

Gaurav Sahu

arXiv preprint arXiv:1904.06022, "16 " # "oct" 2019

Abs

Identifying emotion from speech is a non-trivial task pertaining to the ambiguous definition of emotion itself. In this work, we adopt a feature-engineering based approach to tackle the task of speech emotion recognition. Formalizing our problem as a multi-class classification problem, we compare the performance of two categories of models. For both, we extract eight hand-crafted features from the audio signal. In the first approach, the extracted features are used to train six traditional machine learning classifiers, whereas the second approach is based on deep learning wherein a baseline feed-forward neural network and an LSTM-based classifier are trained over the same features. In order to resolve ambiguity in communication, we also include features from the text domain. We report accuracy, f-score, precision, and recall for the different experiment settings we evaluated our models in. Overall, we show that lighter machine learning based models trained over a few hand-crafted features are able to achieve performance comparable to the current deep learning based state-of-the-art method for emotion recognition.

2018

EMNLP

Free as in Free Word Order: An Energy Based Model for Word Segmentation and Morphological Tagging in Sanskrit

Amrith Krishna, Bishal Santra, Sasi Prasanth Bandaru, and 4 more authors

In Proceedings of EMNLP, Oct 2018

Abs DOI Supp

The configurational information in sentences of a free word order language such as Sanskrit is of limited use. Thus, the context of the entire sentence will be desirable even for basic processing tasks such as word segmentation. We propose a structured prediction framework that jointly solves the word segmentation and morphological tagging tasks in Sanskrit. We build an energy based model where we adopt approaches generally employed in graph based parsing techniques (McDonald et al., 2005a; Carreras, 2007). Our model outperforms the state of the art with an F-Score of 96.92 (percentage improvement of 7.06%) while using less than one tenth of the task-specific training data. We find that the use of a graph based approach instead of a traditional lattice-based sequential labelling approach leads to a percentage gain of 12.6% in F-Score for the segmentation task.