learning.understanding.cognition.intelligence.data science


HAMLET  (Human, Animal, and Machine Learning: Experiment and Theory)

HAMLET is an interdisciplinary proseminar series that started in 2008. The goal is to provide behavioral and computational graduate students with a common grounding in the learning sciences. Guest speakers give a talk each week, followed by discussions. It has been an outlet for fresh research results from various projects. Participants are typically from Computer Sciences, ECE, Psychology, Educational Psychology as well as other parts of the campus. Multiple federal research grants and publications at machine learning and cognitive psychology venues have resulted from the interactions of HAMLET participants.

Meetings: Fridays 3:45 p.m. – 5 p.m. Berkowitz room (338 Psychology Building)

Subscribe/Unsubscribe: HAMLET mailing list

This semester we are trying a new format for HAMLET. Speakers pursuing research at the intersection of computation and human behavior will give a short (10-15 minute) introduction/outline/description of a current research project or problem, with the aim of sparking collaborative discussion in the group about the project.

How would people from your field approach it?
Are there existing tools for solving the problem?
What is already known about the domain?
What kind of data are involved?
How does the research connect to real-world issues?

Brainstorm sessions will be interleaved with standard talks on more mature work throughout the semester.

Spring 2021 Schedule: Meetings are online via zoom please subscribe to the HAMLET mailing list to get the meeting link. 

2/5 James Evans: Accelerating Science with the Most and Least Human Artificial Intelligences

Abstract: This talk is work done by both Jamshid Sourati, Santa Fe Institute and James Evans, University of Chicago and Santa Fe Institute.

Data-driven artificial intelligence models fed with published scientific findings have been used to create powerful prediction engines for scientific and technological advance, such as the discovery of novel materials with desired properties2–4 and the targeted invention of new therapies and vaccines5–7. These AI approaches ignore the distribution of human prediction engines, however, who continuously alter the landscape of discovery and invention. As a result, AI hypotheses intentionally compete with human experts2, failing to complement them for punctuated collective advance. Here we show that incorporating the distribution of human expertise into self-supervised models by training on inferences cognitively available to experts facilitates construction of (1) artificial human intelligences that dramatically improve the prediction of future human discoveries and inventions; and (2) artificial “alien” intelligences that identify scientifically and technologically promising directions unlikely to be pursued without intervention. Including expert-awareness into models that propose valuable energy-relevant materials increases the precision of thermoelectric, ferroelectric and photovoltaic materials predictions by 62%, 140%, and 81%, respectively. By incorporating experts-awareness into proposals for repurposing thousands of drugs across 100 relevant diseases, we increase prediction precision by 43%, and for COVID-19 vaccines examined in clinical trials 260%, exceeding the performance of models that take detailed experimental information into account. We show how these models succeed: the density of experts that link potential scientific and technological inferences predict the likelihood of future human predictions and the scientists who will make them, allowing expert-aware AIs to leapfrog experts on the path to likely discoveries and inventions. By avoiding the crowd, however, our models can generate promising “alien” hypotheses very unlikely to be considered without them. These models also suggest opportunities to reformulate science education for discovery, resisting disciplinary herding and channeling the next generation to strategic locations along the evolving frontier of knowledge.

2/12 Gary Lupyan: Discussion of the article “On the Measure of Intelligence” by Francois Chollet

Please read this article: https://arxiv.org/abs/1911.01547 as Gary will be facilitating a discussion on this article.

2/19 Hayley Clatterbuck: Learning Incommensurate Concepts

Abstract: Jerry Fodor famously argued that it is impossible to learn new concepts that cannot be defined in terms of those concepts with which one began. Hence, it follows that concepts such as avocado or electron cannot be learned. More generally, a long tradition of philosophical arguments have purported to show that there cannot be rational transitions between incommensurable sets of concepts. Given the data of radical concept change in both science and cognitive development, explaining how such concepts can be learned has been an important area of research. I argue that several prominent responses to Fodor’s problem are promising but incomplete. I buttress these responses by showing how modern machine learning algorithms manage to learn incommensurate concepts and draw lessons for understanding the structure of human concepts and how they can be learned.

If you missed this or loved it so much, you’d like to watch it again here is the URL and access passcode:
Start Time : Feb 19, 2021 03:44 PM
Meeting Recording: HAMLET – Hayley Clatterbuck

Access Passcode: 2b3|!2b3

2/26 Tim Rogers: Revisiting conceptual generativity in deep neural network

Do deep neural networks solve conceptual generativity? Tim will be speaking on the topic of conceptual generativity—whether contemporary neural network models offer a new solution to age-old questions about the human ability to imagine and reason about objects far outside the scope of their direct experience. This is work in progress described in further detail below and he hopes to spark a discussion with the group rather than argue for a solution!

Abstract: Fodor famously observed that the composition of two concepts often supports inferences/extensions quite different from those supported by each concept independently. For instance, a typical PET is brown and furry; a typical FISH is greenish and about 8 inches long and found in a lake or ocean; but a typical PET FISH is small, orange, and found in an aquarium. The composition of PET and FISH thus refers to a concept with properties not present in either component. Compositional phenomena of this kind were difficult to understand under symbolic computational approaches to conceptual representation. Fodor concluded that composition was unlearnable and consequently that all concepts must, in some sense, be innate. Recently several groups in machine learning have been investigating the compositional capabilities of deep generative neural network models trained on visual stimuli. I am interested in understanding whether these models provide a new computational mechanism for understanding conceptual composition, and generativity of conceptual knowledge more broadly.

In this talk I will consider what kinds of model behaviors might “count” as evidence of conceptual generativity, by first considering what kinds of human behaviors are held up as examples in various thought experiments on this topic. I will then describe some simulations with different model architectures, comparing and contrasting the degree to which they show comparable behaviors. These results suggest that some well-known approaches (B-VAE and other “disentanglement”-based models) may be less promising than the literature suggests, but that others (deep convolutional conditional auto-GANs!?) show surprisingly broad patterns of behavior that might provide a template for understanding conceptual generativity in human mind and brain.

NB this is work in progress, we are stuck at an exciting point, and I welcome discussion on any of these ideas.

3/5 Amy Cochran: Optimizing Mobile Delivery of Dialectical Behavior Therapy

Most Americans with mental illness do not seek treatment, which is disappointing given the remarkable strides being made in mental health care. One such success is dialectical behavior therapy (DBT), a transdiagnostic treatment program for improving emotion regulation and reducing non-suicidal self-injury. Yet, many barriers prevent people from accessing DBT. Mobile health promises to overcome these barriers. It also can adapt the delivery of care according to how the person is doing in-the-moment. The challenge is acquiring the evidence – at the level of a randomized control trial (RCT) – to learn how to deliver and adapt an mHealth intervention. In this talk, I would like to discuss and brainstorm how we go about acquiring this evidence for mobile DBT.

3/12 Jerry Zhu & Martina Rau: Digitally Inoculating Graph Viewers against Misleading Bar Charts

Bar charts are common visual tools used to convey statistical information. Even though bar charts are effective in making abstract concepts more accessible, poorly-designed bar charts – whether designed intentionally or unintentionally – can easily mislead the viewer. For example, a poorly-designed bar chart may only present part of the effective range on the vertical axis. This exaggerates the contrast among bars, leading an unsuspecting graph viewer to wrong conclusions. More broadly, misrepresentation in data visualization is becoming an increasing societal problem contributing to daily misinformation. We will present a computational and cognitive solution to this problem. Our idea is to “digitally inoculate” viewers by showing them a few dozen carefully designed bar charts that are misleading, together with guidance on why these bar charts are misleading. We then test whether the viewers are immune to similarly misleading bar charts in the future. Importantly, we use neural networks and cognitive models to optimize the “vaccine” (i.e., the design of those few dozen bar charts). Our experiment shows that digital inoculation can help viewers not be fooled by such misleading graphs in the future. We discuss how we plan to build on these findings in future research.

3/19 Greg Henselman-Petrusek: What is a neural representation, and what do we really mean when we say that two processes “share” one?

Greg Henselman-Petrusek, formerly of the Cohen lab at Princeton and currently at the Center for Topological Data Analysis at Oxford (see: http://gregoryhenselman.org/). Greg’s work applies mathematical analysis to neural network models to understand critical issues in cognitive control/executive function–that is, in our ability to flexibly adapt, re-shape, and re-use mental representations across many different tasks.
Artificial systems currently outperform humans in diverse computational domains, but none has  achieved parity in speed and overall versatility of mastering novel tasks.  A critical component to human success, in this regard, is the ability to redeploy and redirect  data passed between cognitive subsystems (via abstract feature representations) in response to changing task demands.  Traditionally, we say that a representation is “shared” if it is used by multiple different subsystems.  However, formalizing the idea of a shared representation is challenging, especially with distributed nonlinear neural coding.  This work presents a simple but effective approach.  In experiments, the model robustly predicts behavior and performance of multitasking networks on natural language data (MNIST) using common deep  network architectures.  Consistent with existing theory in cognitive control,  representation structure varies in response to (a) environmental pressures for representation sharing, (b) demands for parallel processing capacity, and (c) tolerance for crosstalk.  We will discuss these ideas in terms of a feed-forward network with a single hidden layer.

3/26 No HAMLET for LUCID Bias Training

4/2 No HAMLET for University Spring Holiday

4/9 Judy Fan: Cognitive tools for making the invisible visible

How does the human mind transform a cascade of sensory information into meaningful knowledge? Traditional approaches focus on how people process the data provided to them by the world, yet this approach leaves aside some of the most powerful tools humans have to actively reformat their experiences, including the use of physical media to externalize their thoughts by drawing or writing. My lab aims to “reverse engineer” the core mechanisms by which employing such cognitive tools enable people to learn and communicate more effectively. Our recent work focuses on sketching, one of our most basic and versatile tools, because it also represents a key challenge for understanding how multiple cognitive systems interact to support complex, natural behaviors. This talk will highlight our recent progress, as well as open research questions in this domain.

4/16 Ashique KhudaBukhsh: Political Polarization and International Conflicts through the Lens of Natural Language Processing

Ashique KhudaBukhsh is currently a Project Scientist at the Language Technologies Institute, Carnegie Mellon University (CMU) mentored by Prof. Tom Mitchell. Prior to this role, he was a postdoc mentored by Prof. Jaime Carbonell at CMU. His PhD thesis (Computer Science Department, CMU, also advised by Prof. Jaime Carbonell) focused on distributed active learning. His current research lies at the intersection of natural language processing (NLP) and AI for Social Impact. In this field, he is interested in analyzing globally important events in South East Asia and developing methods for noisy social media texts generated in this linguistically diverse region. His other broad research focus is US politics; in this area, his research involves devising novel methods to quantify, interpret and understand political polarization.

Abstract: In this talk, I will summarize two broad lines of NLP research focusing on (1) the current US political crisis and (2) the long-standing international conflict between the two nuclear adversaries India and Pakistan.

The first part of the talk presents a new methodology that offers a fresh perspective on interpreting and understanding political polarization through machine translation. We begin with a novel proposition that two sub-communities viewing different US cable news networks are speaking in two different languages. Next, we demonstrate that with this assumption, modern machine translation methods can provide a simple yet powerful and interpretable framework to understand the differences between two (or more) large-scale social media discussion data sets at the granularity of words.

The second part of the talk seeks to examine what we term as hostility-diffusing, peace seeking hope speech in the context of the 2019 India-Pakistan conflict. In doing so, we tackle several practical challenges that arise from multilingual texts and demonstrate how novel methods can effectively extend linguistic resources (e.g., content classifier, labeled examples) from a world language (e.g., English) to a low-resource language (e.g., Hindi). To this end, we show two different approaches — one relying on code switching and the other relying on unsupervised machine translation — which achieve substantial improvement in detecting Hindi hope speech under low-supervision settings.

4/23 Becca Willett: Deep Learning Techniques for Inverse Problems in Imaging

Recent work in machine learning shows that deep neural networks can be used to solve a wide variety of inverse problems arising in computational imaging, where training data is essentially used to learn a regularization function. We explore the central prevailing themes of this emerging area and present a taxonomy that can be used to categorize different problems and reconstruction methods. We also discuss the trade-offs associated with these different reconstruction approaches, caveats and common failure modes, plus open problems. This is joint work with Davis Gilton and Greg Ongie.

4/30 Karen Schloss & Kushin Mukherjee: Context Matters: A Theory of Semantic Discriminability for Perceptual Encoding Systems

People’s associations between colors and concepts influence their ability to interpret the meanings of colors in information visualizations. Previous work has suggested such effects are limited to concepts that have strong associations with colors. However, although a concept may not be strongly associated with any colors, its mapping can be disambiguated in the context of other concepts in an encoding system. We articulate this view in Semantic Discriminability Theory, a general framework for understanding conditions determining when people can infer meaning from perceptual features. Semantic discriminability is the degree to which observers can infer a unique mapping between visual features and concepts.

Semantic Discriminability Theory posits that the capacity for semantic discriminability for a set of concepts is constrained by the difference between the feature-concept association distributions across the concepts in the set. We define formal properties of this theory, and test its implications in three experiments. The results show that the capacity to produce semantically discriminable colors for sets of concepts was indeed constrained by the statistical distance between color-concept association distributions. Moreover, people could interpret meanings of colors in bar graphs insofar as the colors were semantically discriminable, even for concepts previously deemed “non-colorable”. The results suggest that colors are more robust for visual communication than previously thought.

Karen Schloss and Kushin Mukherjee will be presenting joint work from the authors: Kushin Mukherjee, Brian Yin, Brianne Sherman, Laurent Lessard, Karen Schloss

5/7 Will Cox: Harnessing Scientific Approaches to Create Lasting, Meaningful Social Impact: Changing Social Systems by Empowering Individuals

Anti-bias trainings (a.k.a. anti-racism trainings, diversity and inclusion trainings) proliferate in the public realm. However, when they are assessed empirically, the evidence indicates that they at best do nothing and at worst create more bias-related issues. Creating real, lasting change requires developing training and intervention approaches based on solid scientific evidence that are assessed in rigorous experimental trials. The bias habit breaking intervention was the first, and remains the only intervention that has been shown experimentally to produce long-term reductions in bias and improvements in diversity and inclusion. Across 13 years of in-lab and real-world randomized-controlled experiments, this intervention has shown remarkable replicability and long-term effectiveness, with effects lasting up to at least 2-3 years. The intervention’s approach is designed to empower people to confront and reduce bias within themselves and within the social systems they inhabit. In the present talk, I will 1) discuss the need for evidence-based approaches to address inequality and systemic injustice, 2) unpack the rationale and evidence base for the development of the habit-breaking training, and 3) review evidence demonstrating the effectiveness of the habit-breaking training. When scientific approaches are applied to study and improve social problems, we can create meaningful, real-world, long-lasting changes. As individuals, we should advocate for evidence-based interventions to ensure an ethical implementation of the values related to equity and inclusion. Scientific approaches can be powerfully translated to enable individuals to positively impact their world in many different contexts.


Fall 2020 Schedule:

HAMLET this semester will be virtual, make sure that you are subscribed to the HAMLET list in order to get the link to the meeting.

If there is any silver lining to our current national struggles, it is that it is easier than ever to remotely bring in speakers from across the globe.

10/2 Paul Kantor with Vicki Bier, Gary Lupyan and Jerry Zhu

Paul Kantor will be discussing a recently funded in-development project, comparing rule-learning by humans and machines.

Title: Human and Machine Learning: Searching for Differences

Abstract: We are using “rule finding games” to look for anomalous pairs of rule classes. A pair is anomalous if the class that is more difficult for humans is easier for machines.  The study will use a “table clearing” game, in which figures with varying shape, color, and position are displayed on a screen with four “buckets” in the corners. The rules determine the order in which the objects are to be placed into buckets, and the target placement for each object. Examples might be “circles in bucket A; all others anywhere” or “put each object in its closest bucket, farthest first, and decreasing to the one whose closest bucket is nearest.” The space of possible rules is enormous, permitting stochastic rules, history-dependent rules, and rules depending on any aspect of the object that is visible to humans.  I will talk about how we got started, and a bit about where we hope to arrive, and even less about what we already know.

10/9 Sebastian Musslick: On the artional bounds of cognitive control

Sebastian Musslick, a scientist in Jonathan Cohen’s lab at the Princeton Neuroscience Institute. Sebastian’s work involves mathematical and simulation analyses of neural network models to understand a central aspect of human cognition: why can’t we do many different cognitive tasks in parallel, and how is it that we sometimes can? Put differently: Why can we walk and chew gum, but can’t text and drive?

Title: On the Rational Bounds of Cognitive Control

Abstract: One of the most remarkable features of the human brain is its ability to adapt behavior in a changing world, e.g. avoiding to touch one’s face during elevated risk of viral infection. Mechanisms underlying this function are summarized under the term cognitive control, and are engaged across all the higher mental faculties that distinguish us as
humans, including language, reasoning and problem solving. Yet, humans are strikingly limited in how many control-demanding tasks they can perform at the same time (e.g. reading a document while listening to a colleague). These limitations became a fundamental premise for most general theories of human cognition. However, none of these theories
provides a rational explanation for why neural systems, such as the human brain, would be limited in the number of control-demanding tasks it can perform in parallel. In this talk, I argue that such constraints on multitasking capability may result from a rational adaptation to a fundamental computational dilemma in neural architectures. Using neural network simulations and behavioral experiments I demonstrate that neural architectures are subject to a tradeoff between learning efficacy, that is promoted through the use of shared task representations, on the one hand, and multitasking capability, that is achieved through the separation of task representations, on the other hand. These results suggest that the limited capability to execute multiple tasks in parallel may reflect a preference of the neural system to learn tasks more quickly. I conclude by suggesting that the tradeoff between shared and separated representations, and its interaction with learning, represent a fundamental principle of adaptive network architectures that underlies and shapes all domains of psychological function, from perception and inference to task execution, and extends equally to artificial systems.

10/23 Ramya Vinayak: Crowdsourced Clustering: Query design and Robust Algorithms

Ramya Vinayak has been doing some exciting work using crowdsourcing of human judgments combined with insights from adaptive sampling to efficiently estimate human-perceived structure in data. Ramya is a new faculty member in ECE and her work dovetails very nicely with work on the NEXT system and other efforts from both computational and behavioral sciences to develop contemporary techniques for quickly finding structure in human-generated

Title: Crowdsourced Clustering: Query design and Robust Algorithms

Abstract: Crowdsourcing is a popular way of collecting labels for unlabeled data to create training data for supervised learning tasks. Collecting labels via crowdsourcing is a noisy process as the workers are usually not domain experts. This makes it difficult to obtain high quality granular data. For example, the workers can easily say if an animal is a dog or a cat but find it difficult to identify the breed of a dog. To overcome this limitation, we propose clustering answers to easier comparison queries. We introduce triangle query (clustering three images per query) for this crowdsourced clustering task and compare it with edge queries (clustering two images per query).  Via experiments on real datasets collected on Amazon Mechanical Turk, we show that for a fixed budget, our design can double the amount of data collected while simultaneously reducing noise. Another important issue in crowdsourcing is that of budget constraint which puts a limitation on the number of queries one can make. Therefore, we need robust algorithms that can perform clustering on partially observed, noisy data. We analyze convex algorithms based on low-rank plus sparse decomposition of matrices and show that they can reliably find clusters in the presence of noise, outliers and missing data both in theory and in practice.

10/30 Patrick Shafto: A story of (children’s) learning, love (of math), and (minimizing) loss

Pat directs the Cognition and Data Science lab at Rutgers, and is Rutgers Term Chair in Data Science. He is also one of the few people Tim knows who moved from an faculty appointment in a Psychology Department to an appointment in a Math/CS Department–so right up HAMLET’s alley.

Pat’s work focuses on using Bayesian models and other computational approaches in aid of understanding human learning and teaching, especially in educational contexts.

Title: A story of (children’s) learning, love (of math), and (minimizing) loss

Starting from the goal of having more rigorous theories of human and machine learning, I will offer a light critique of the state of the art in human and machine learning. I will then introduce a new, rather old, mathematical approach that connects child development, machine learning, and pure mathematics. The technical parts are based on a new paper “A mathematical theory of cooperative communication” to appear in NeurIPS this year.

11/6 Siddharth Suresh & Emily Ward

Title: Visual ensemble representations emerge in Deep Neural Networks trained for object recognition

Becoming aware of the world seems simple and straightforward — but we frequently see the world in a way that differs from how it actually is. We may fail to see a car turning right in front of our eyes, while still feeling like what we see is extremely rich and detailed. How can we perceive so much richness when we are susceptible to such failures of awareness? One possibility is that we perceive ensemble properties of scenes, without being aware of the many individual features. Humans are able to quickly pool information from across many individual objects to perceive ensemble properties, like the average size of objects, the color diversity of objects, or even the gender ratio in a group of faces. Are such ensemble properties also present in artificial visual systems, such as Deep Convolutional Neural Networks? And if so, might these networks also be useful for understanding visual awareness, in addition to their utility in understanding object recognition? Here, we determined whether the ensemble property of color diversity emerges in a network pre-trained to recognize natural objects (using ImageNet). We presented the network with new images that were completely different from its training set: images of letter arrays containing four colored consonants on a black background. The color of each letter within an array was drawn either from a broad sample of colors (high color diversity) or a randomly selected range of adjacent colors (low color diversity). We tested whether a ResNet50 neural network could distinguish high vs. low color diversity arrays by using the activations from different layers as input to a linear classifier (SVM). We found that the network was highly accurate at identifying both color and color diversity. This demonstrates the presence of ensemble representations for the first time in deep neural networks, and also suggests that ensemble perception of multiple objects can emerge even in a system only trained to recognize individual objects. We will discuss how the emergence of ensemble representations in neural networks may make such networks suitable models for failures of visual awareness.

11/13 Gary Lupyan: Can telepathy work?

Science fiction has long used telepathy as a method of efficient communication. The common trope is an advanced alien species communicating with humans telepathically without, of course, sharing a language. Several companies, including Elon Musk’s Neuralink project, aim to make this science fiction a reality. A common assumption in both fictional and aspirational narratives of telepathy is that natural language is slow, imprecise, and mendacious. By bypassing language, a direct mind-to-mind connection is thought to make communication faster, more precise, and truthful.
But could it work? Rather than focusing on the technological aspects of the question, I am going to focus on the psychological. All successful communication requires the signaler and the recipient to agree on a set of mutually understood signals. Linguistic communication fulfills this goal through a learned, shared vocabulary. For telepathy (at least the sci-fi kind) to work requires a shared nonlinguistic vocabulary — a mentalese — that can be used for communication. Do we, in fact, have such a vocabulary? How can we find out?

11/20 Jackie Fulvio: Introducing a new Gender Citation Balance Index web tool for academics

It has recently been reported that in five broad-scope neuroscience journals, citation rates for papers first- and/or last-authored by women are lower than would be expected if gender was not a factor in citation choices (Dworkin et al., 2020). Given the important implications that such underrepresentation may have on the careers of women researchers, it is important to determine whether this same trend is true in subdisciplines of the field, where interventions might be more targeted. We extended the analyses carried out by Dworkin et al. (2020) to citation patterns in the Journal of Cognitive Neuroscience (JoCN) and determined that the reference sections of papers published in JoCN during the last decade or so included women-led papers at a level considerably lower than the expected base rate given the journal’s authorship. Furthermore, this pattern of citation imbalances has been characteristic of the reference sections in all author-gender groups (i.e., “MM,” “MW,” “WM,” and “WW”), thereby implicating systemic factors. These results contribute to the growing body of evidence that intentional action is needed to address inequities in the way that we carry out and communicate our science. To that end, I developed a web tool that went “live” in late October, 2020, that accepts an author’s reference list and returns the categorical gender breakdown. I will introduce this tool with a deep dive into how it works. I’ll then discuss how we had envisioned this tool would be used in cognitive neuroscience and give a brief report about useability and feedback, successes and failures, to date. Finally, I’ll discuss possible uses and extensions of the tool beyond its initial application in cognitive neuroscience.

11/27 Thanksgiving

12/4 James Evans


Fall 2019 Schedule:

HAMLET this semester will have a mini-series on neural networks.

9/27 Gary Lupyan lead a group discussion on what does it mean to understand a neural network?

What does it mean to understand what a neural network is doing? How do people in different disciplines–cognition, neuroscience, machine learning–think about or answer this question? To orient the discussion, please have a look at the following recent paper by Tim Lillicrap (machine learning scientist at Deep Mind) and Konrad Kording (computational neuroscientist at Penn):


We are very interested to know whether different disciplines even agree on what “interpretability” means, and whether there is useful cross-disciplinary progress to be made.

10/4 Yingyu Liang

Title: Robust Attribution Regularization
Abstract: An emerging problem in trustworthy machine learning is to train models that produce robust interpretations for their predictions. We take a step towards solving this problem through the lens of axiomatic attribution of neural networks. Our theory is grounded in the recent work, Integrated Gradients (IG), in axiomatically attributing a neural network’s output change to its input change. We propose training objectives in classic robust optimization models to achieve robust IG attributions. Our objectives give principled generalizations of previous objectives designed for robust predictions, and they naturally degenerate to classic soft-margin training for one-layer neural networks. We also generalize previous theory and prove that the objectives for different robust optimization models are closely related. Experiments demonstrate the effectiveness of our method, and also point to intriguing problems which hint at the need for better optimization techniques or better neural network architectures for robust attribution training.

This is joint work with Jiefeng Chen, Xi Wu, Vaibhav Rastogi, and Somesh Jha.

10/11 Rahul Parhi: Neural Networks Learn Splines

Where: State Room (4th floor) in Memorial Union

We develop a general framework based on splines to understand the interpolation properties of overparameterized neural networks. We prove that minimum “norm” two-layer neural networks (with appropriately chosen activation functions) that interpolate scattered data are minimal knot splines. Our results follow from understanding key relationships between notions of neural network “norms”, linear operators, and continuous-domain linear inverse problems.

10/18 Larissa Albantakis: Causation vs prediction: Understanding *why* an agent did what it did

As part of our mini-series on neural networks, Larissa Albantakis from the Wisconsin Institute for Sleep and Consciousness will be speaking on the topic of causation vs prediction in understanding the
behavior of artificial agents. Larissa has training in physics and
computational neuroscience, and has developed computational and
neural-network-based approaches to difficult problems in cognition and
neuroscience, including contributions to the integrated-information approach to consciousness and to theories of human decision-making.

When an agent interacts with its environment, we are often interested in  why the agent performed a particular action. Due to recent advances in the field of artificial intelligence, there is a growing realization that the ability to predict what a system is going to do does not equal understanding how or why it behaves in a certain way, not even in hindsight. This is demonstrated particularly well by recent computational studies involving simulated artificial agents with minimal cognitive architectures, whose behavior can easily be predicted. Yet, understanding what caused the agent to perform a particular action typically requires extensive additional analysis and cannot be addressed in purely reductionist or holistic terms. Finally, the same action can be performed for different reasons and with different levels of autonomy. How can we distinguish reflexes from autonomous actions in mechanistic terms?

10/25 Claudia Solis-Lemus: Deep learning methods to predict antibiotic-resistance in microbial data

Abstract: One of the golden standards of precision medicine is the accurate prediction of an individual’s antibiotic response based on host’s as well as bacteria’s genomic data, especially for severely infectious pathogens like Staphylococcus aureus and Pseudomonas aeruginosa. I will present the recent work on deep learning techniques to predict antibiotic-resistant traits on microbial genomic data. I will focus mostly on the challenges that users normally face when trying to fit deep learning models from the choice of architecture and other parameters to small sample sizes with missing and sparse data.

11/1 Mike Zinn

Where: Union South, Landmark Room

Title: Toward a Robotically-Guided Diagnostic and Rehabilitation Paradigm for the Treatment of Motor Skill Deficiencies in Children with Autism Spectrum Disorder

Many children with autism spectrum disorder (ASD) struggle with motor impairments and these motor impairments have been found to predict difficulties with independent living skills and the severity of core social communication impairments. While many of us take for granted how intertwined our motor skills are with our social and cognitive abilities, motor difficulties in autism may drastically limit a child’s ability to explore the world, competently interact with others, and participate in activities with peers. As such, motor skill therapy can offer a unique and under-utilized therapeutic approach to improve the independent living skills and social skills of children with autism.

One challenge in training motor skills in ASD, is that individuals with autism have a unique motor learning style that necessitates utmost simplicity and consistency in presentation, which is difficult for a human therapist to enact with fidelity. In addition, diagnosis and assessment is commonly performed using motor skill metrics where the assessment relies on the expertise and, to some degree, qualitative judgment of the therapist. As such current approaches for assessment of motor skills are time consuming, prone to error, and can result in widely varying diagnosis depending on the skill and experience of the assessing clinician.

Is there a more direct, quantitative approach for both assessment and treatment?   When considering current assessment methods, they focus on measuring the expression of the impairment, in terms of specific task skills, will little to no consideration of underlying physical and/or neurological characteristics. What if these could be measured, using robotic rehabilitation to excite and measure the underlying dynamic system characteristics while also providing the consistency in presentation required when working with children with ASD.

Could accurate measurements be used to develop mathematical models and dynamic system descriptions of the underlying source of the motor skill impairment?  Would these mathematical models relate to observed motor skill deficiencies and would this description provide a window into the mechanisms of the motor impairments?  Would it provide a possible avenue through which to target and assess treatment approaches by seeking to modify the structure or parameters of the dynamic system models associated with motor skill deficiency?  To answer these questions, we need to understand the relationship and connections between the measurements of physical interaction between human patient and robot, assessment of motor skill proficiency, coupled with ASD diagnostic assessment.

The talk will present preliminary human-study data and discuss the host of challenges that remain. The work presented here is from an on-going collaboration between Brittany Travers (Kinesiology), Andrea Mason (Kinesiology), and Mike Zinn (Mechanical Engineering)

11/8 Peter Adamczyk and Alexander Dawson-Elli
Title: The Challenge of Eliciting and Modeling Motor Learning

In addition to explicit knowledge, the brain and other neural structures must learn and apply knowledge of how to control the body to move and accomplish tasks. This Motor Learning, and its short-term cousin Motor Adaptation, have distinct characteristics, such as exponential convergence over time, specific adaptation patterns to different conditions, and circumstances in which it can and cannot be accomplished. Motor Learning has been used as a guide for developing rehabilitation strategies for persons with neuromotor injury such as stroke, but with limited success. One of the challenges is developing conceptual and computational models of the state of the neural motor control system and using these to predict outcomes and adapt rehabilitative tasks. This seminar will provide a primer on principles of motor learning, and present some challenges in modeling that we hope HAMLET participants can help address.

11/15 Bilge Mutlu & David Porfirio

Title: Computational Tools to Support the Design of Artificial Agents

Abstract: Artificial characters—from chatbots to social robots—are expected to follow human norms and mechanisms of communication, and designers of these characters are expected to encode such norms and mechanisms into their systems. How can designers formalize this information into computational representations? How can they ensure that systems will follow these specifications in their interactions with people? How can these systems customize and adapt their behaviors to the needs, expectations, and preferences of their users? In this talk, we will give an overview of a new research program that bridges interaction design and formal methods in the context of designing artificial agents toward building design support tools. We will present our vision for such tools, illustrate our approach with three tools we have built, and conclude with a discussion of our future work.

11/22 Yin Li

Title: Towards Computational Imaging of Daily Activities–From Body Movements to Human-Object Interactions

Abstract: Major progress has been made in visual sensing and computer vision towards human body pose estimation. This has enabled novel means of measuring our purposive body movements in the context of daily activities using minimally intrusive cameras. The ability to measure body motion in turn provides valuable insights to assess our physical activities. In this talk, I will describe our ongoing efforts on sensing and modeling human body movements for understanding human-object interactions. First, I will present our work on future interaction anticipation from a first person video captured by a body-worn camera. We demonstrate that modeling of the camera wearer’s hand trajectory leads to more accurate prediction of actions in the near future. Second, I will discuss our work on estimating the weight of an object that a user is lifting from a third person video. The key hypothesis is that the lifting load is characterized by distinct patterns of a user’s joint trajectory. I will describe our pilot study and preliminary results. Finally, I will briefly introduce our work on developing wrist-worn visual sensor for continuous 3D finger pose tracking. Putting things together, I will discuss the new opportunities for imaging our daily activities.

11/29 Thanksgiving – no HAMLET

12/6 Michael Gleicher

Title: Dumb Robots for Smart People

Abstract: Even with advances in intelligence and learning, there are still places where people need to control robots. Examples include tele-operation for automation resistant tasks and providing demonstrations for robot learning. In such situations, direct interfaces, where a person’s movements are mapped to the robot, can be advantageous. In this talk, I will describe our recent efforts to create better direct control interfaces. First, I will discuss our work on arm-scale tele-operation systems based on mimicry, the idea of making the system feel as if it is mimicking a users’ movement. I will describe how we address the basic challenges of mapping movements to robots which allows us to see the benefits of direct control. I will then describe extensions that use elements of shared control to address challenges in watching what the robot is doing as well as performing two handed tasks. Second, I will discuss our work on developing input devices that permit people to demonstrate tasks to robots in a natural, yet constrained, manner. I will show how our ability to better instrument demonstrations provides information that allows us to infer the mechanical properties of the actions, which allows for more informed use of the demonstrations. With both projects, I will discuss the opportunities for combining direct control with intelligence to create even better performance.

Spring 2019 Schedule:

2/8 John Curtin, Psychology

Brainstorm: Digital phenotyping with (somewhat) big data for harm reduction in alcohol and other drug use disorders

John has been applying machine learning methods to Facebook and geo-tracking data to understand and predict patterns of relapse in
people recovering from addiction. He will briefly characterize issues that motivate the research, the kinds of data his group is working with,
some of the work they have already done and some of the challenges they are currently facing. The discussion will focus on generating ideas for moving the project forward.

2/15: Data blitz: Current research in cognition, perception, and cognitive neuroscience (special event at the WID)

2/22: Paula Niedenthal, Psychology

Brainstorming connections between emotion, cognition and datascience

Paula will initiate a session concerning ways that scientists in machine learning/data science might collaborate with scientists working to understand human affective cognition. Read about some of her recent work here: Science Daily

3/1: Dimitris Papailiopoulos (ECE, WID)  LOCATION CHANGE: Union South (look at the TTIU for the room)

Dimitris Papailioupolos will spark a discussion assessing whether principles of human cognition can guide us toward more robust machine learning.

3/8 Alyssa Adams, VEDA Data Solutions

Emergence, evolution and video games

Alyssa Adams, a data scientist working at Madison startup VEDA Data Solutions. Alyssa’s work focuses on understanding what makes living systems different from non-living systems, and in particular the mechanisms that underlie emergence in social and biological systems. Toward these questions she applies computational approaches to data generated by people interacting in video games–so if you like emergence, math, human behavior and video games, you should really come. Alyssa is also a great person to talk to if you are interested in how data science is being applied in the wild, or what it is like to move from an academic to an industrial career path.

3/22  Spring Break

3/29 Karen Schloss, Psychology & WID & Laurent Lessard, WID/ECE

Using machines to learn how humans map colors to concepts

Karen Schloss and Laurent Lessard with lead a brainstorming session focusing on the use of machine learning to understand a central aspect of human cognition: what explains the conceptual inferences and associations that people draw from simple visual features, such as colors? Karen and Laurent will briefly set up the problem, describing some interesting phenomena and prior work, then will lead an interactive discussion on ways that machine learning might be developed or applied to solve key questions.

4/5 Jon Willits, Psychology at University of Illinois

A Two-Process Model of Semantic Knowledge

Jon’s work has connected large-corpus NLP-based approaches to word meaning with cognitive theories of language. He will present new work on this topic, suggesting a two-stage model of semantic processing with implications for both human language and the design of artificial knowledge systems.

Abstract: Computational models of semantic memory, and machine learning approaches to knowledge-based systems, have traditionally proposed that there is a single semantic memory – one set of factual knowledge that is
used for all activities. There are a number of problems (both theoretical and empirical) with this view. In this talk, I will propose an alternate, two-process model. Process 1 consists of a general, passive, statistical-learning system that acquires knowledge of general, context-insensitive associations and similarities in the environment.
Process 2 consists of task- or goal-specific subsystems, where stored semantic representations are changed in ways that optimize their use for specific situations. I will show how this model addresses a number of thorny theoretical and empirical issues in the study of semantic memory and building artificial knowledge systems.

4/12 Ari Rosenberg, Neuroscience

Inferring whole brain connectivity

Knowing the interconnections between brain regions is important to understand how neural functions are implemented by distributed networks, and how different networks share information. Diffusion tensor imaging (DTI) provides a technique for estimating anatomical pathways within the brain. Our preliminary attempts to create connectivity matrices for macaque cortex using DTI have proven promising. First, we find a surprisingly strong ability to predict functional connectivity data from other labs. Second, we find evidence of pathways hypothesized to exist based on electrophysiological data. A potential point of discussion is whether graph theory provides appropriate techniques for quantifying brain networks using DTI data.

4/19 Greg Zelinsky, Psychology at Stony Brook University

Predicting goal-directed attention control: A tale of two brain-inspired computational models

The ability to control the allocation of attention underlies all visuospatial goal-directed behavior; there would be no sense in having a goal if there was no way to use it to select visual inputs. He will summarize two recent efforts to computationally model this core mechanism of cognitive control. These models are similar in that both are image computable and that the design of each is inspired by mechanisms used by the attention-control system in the primate brain. But the models also differ in their type of brain inspiration and in their neurocomputational approach. One model adopts a computationally narrow approach but is closely parametrized by known neural constraints (MASC, a Model of Attention in the Superior Colliculus). The other model adopts a more powerful deep neural network approach but is less directly brain inspired (Deep-BCN, a deep network implementation of the widely accepted biased-competition theory of attention control). Together they are intended to span a continuum of brain-inspired model design, ranging from relatively local neural circuits at one end to the broader network of brain areas comprising the primate attention control system at the other.

4/26 Glenn Fung Moo, a machine learning scientist at American Family Insurance

Non-IID Model Learning via Graph Neural Networks

Most of the more commonly used machine learning methods assume that training and test data comes from independent and identically distributed (IID) random variables. However, this assumption is strongly violated when the data points available to learn/test the model are highly interdependent. Two examples of this scenario are: when the data exhibits temporal or spatial correlations and when the task is to learn relative characteristics between data points (e.g., ranking or attribute learning). In this talk we show how emerging ideas in graph neural
networks can yield a solution to various problems that broadly fall under this characteristics. More specifically, we show interesting results for the relative attribute learning problem from images. This setting, naturally benefits from exploiting the graph of dependencies among the different relative attributes of images, especially when only
partial ordering is provided at training time. Our experiments show that this simple framework is effective in achieving competitive accuracy with specialized methods for both relative attribute learning and binary attribute prediction.

5/3 Sarah Sant’Ana, Psychology



Fall 2018 Schedule:

Sep 14, Tim Rogers, Psychology

Investigating the covariance structure of widely-held false beliefs

ABSTRACT: It has become increasingly clear that our society is suffering a crisis of false belief. Across a variety of domains–including science, health, economics, politics, history, and current events–there often exists vociferous disagreement about what ought to be matters of fact. I’d like to know whether cognitive science and machine learning together can help us to understand why this is happening. In prior work we have studied the formation of false beliefs in small well-controlled lab studies with no real-world stakes. In this talk I will describe some preliminary work looking at real false beliefs occurring in the wild. In collaboration with a summer PREP student and colleagues in Psychology and in the School of Journalism, we generated a long list of incorrect but putatively widely-held beliefs spanning many different knowledge domains. Using Amazon Mechanical Turk, we asked people about the degree
to which they endorsed or rejected the belief. We also collected a variety of data about each respondent, including sociodemographic data, political leanings, media trust, IQ, and so on. The aim was to measure how susceptibility to false beliefs pattern across individuals, and to understand what properties of individuals predict susceptibility to which kinds of false beliefs. Our early results provide some provocative evidence contradicting some straight-forward hypotheses about false belief formation, but there is a lot more work to be done. We would like to connect with others interested in these problems from a machine-learning-and-social-media perspective, and in particular would like to consider ways of using social network analytics to assess the degree to which media consumption may be causing the patterns we observe.

Sep 21, LUCID faculty meeting – no HAMLET

Sep 28, no HAMLET

Oct 5, Jay Patel, Educational Psychology and Ayon Sen, Computer Science

A Novel Application of Machine Teaching to Perceptual Fluency Training, a collaboration between Ed Psych and CS

W will hear from Jay Patel and Ayon Sen, who will present new work applying neural networks and machine teaching to undergraduate chemistry learning—something for everyone!


In STEM domains, students are expected to acquire domain knowledge from visual representations that they may not yet be able to
interpret. Such learning requires perceptual fluency: the ability to intuitively and rapidly see which concepts visuals show and to translate among multiple visuals. Instructional problems that engage students in nonverbal, implicit learning processes enhance perceptual fluency. Such processes are highly influenced by sequence effects. Thus far, we lack a principled approach for identifying a sequence of perceptual-fluency problems that promote robust learning. Here, we describe a novel educational data mining approach that uses machine learning to generate an optimal sequence of visuals for perceptual-fluency problems. In a human experiment, we show that a machine-generated sequence outperforms both a random sequence and a sequence generated by a human domain
expert. Interestingly, the machine-generated sequence resulted in significantly lower accuracy during training, but higher posttest
accuracy. This suggests that the machine-generated sequence induced desirable difficulties. To our knowledge, our study is the first to show that an educational data mining approach can induce desirable difficulties for perceptual learning.

Oct 12, Tim Rogers, Psychology

Data science and brain imaging at UW Madison

Over the summer Tim had conversations with several groups about their work applying machine learning or other data science tools to brain imaging data. These activities are part of a broader trend across the country, with collaborations between neuroscientists, cognitive scientists, and computer scientists driving very rapid innovations in techniques for finding signal in neural data. Many of us are interested in connecting disparate efforts across campus to develop a general knowledge base, community of support, and new collaborations in this field. There is also interest in understanding how work conducted here relates to other high-profile work emerging from other labs.

Tim will present a general overview of the variety of new methods that I am aware of and some of the astonishing results they have uncovered. The overview will be targeted at both cognitive scientists/neuroscientists and members of the machine learning
community. I will also highlight some of the opportunities for new work and collaboration that I am aware of, and hope to initiate a discussion about what we can do to promote these efforts at UW.

Join the new email list to facilitate discussion, collaboration, and information-sharing on these issues.  big-lucid@lists.wisc.edu  (BIG = “Brain-imaging Interest Group”; LUCID is the broader community interested in learning, understanding, cognition, intelligence and data-science)

Oct 19, LUCID faculty meeting – no HAMLET

Oct 26, no HAMLET

Nov 2, Jerry Zhu, Computer Science

Discuusion: Can Machine Teaching Influence How Humans Do Regression?

Jerry is interested in discussing with cognitive scientists possible relationships between a hot topic in machine learning–data
poisoning, and defences against data poisoning–and approaches to human learning and teaching. Can central concepts from this area of study in machine learning help or hinder human learning?

How do people make numerical predictions (e.g. weight of a person) from observed features (height, fitness, gender, age, etc.)?  We may assume that they learn this from some sort of regression in their head, applied to training data they receive.  Can machine teaching help them learn better predictions by being careful about the training examples we give them?   Or to mess them up intentionally?  This is equivalent to ‘training data poisoning attacks’ in adversarial machine learning.   I will explain how this is done with control theory under highly simplified, likely incorrect assumptions on human learning.  This is where we need the cognitive science audience to participate in the discussion, and together we will explore the research opportunities on this topic.

Nov 9, Data Blitz

75 minutes, 9 talks, a sampling of current research at UW connecting data science and human behavior

This HAMLET will feature a cross-disciplinary data-blitz. We will hear a series of 5-minute talks on topics at the intersection of
machine learning, data science, and human behavior. This is a great opportunity to hear about some of the projects happening on campus now that you may want to get involved with.

Can neural networks help you learn chemistry? How do Trumpian false beliefs differ from other partisan false beliefs? These questions any many more will be addressed in this data blitz!

Nov 16, Gary Lupen

How translateable are languages? Quantifying semantic alignment of natural languages.

Do all languages convey semantic knowledge in the same way? If language  simply mirrors the structure of the world, the answer should be a qualified “yes”. If, however, languages impose structure as much as reflecting it, then even ostensibly the “same” word in different
languages may mean quite different things. We provide a first pass at a large-scale quantification of cross-linguistic semantic alignment of approximately 1000 meanings in 70+ languages. We find that the translation equivalents in some domains (e.g., Time, Quantity, and Kinship) exhibit high alignment across languages while the structure of other domains (e.g., Politics, Food, Emotions, and Animals) exhibits substantial cross-linguistic variability. Our measure of semantic alignment correlates with phylogenetic relationships between languages and with cultural distances between societies speaking the languages, suggesting a rich co-adaptation of language and culture even in domains of experience that appear most constrained by the natural world.

Nov 23, Thanksgiving – no HAMLET

Nov 30, Emily Ward, Psychology

Dec 7, Priya Kalra, Educational Psychology





Spring 2018 Schedule:

Feb 2, Blake Mason

Title: Low-Dimensional Metric Learning with Application to Perceptual Feature Selection

Abstract: I will discuss recent work investigating the theoretical foundations of metric learning, focused on four key topics: 1) how to learn a general low-dimensional (low-rank) metrics as well as sparse metrics; 2) upper and lower (minimax) bounds on the generalization error; 3) how to quantify the sample complexity of metric learning in terms of the dimension of the feature space and the dimension/rank of the underlying metric; 4) the accuracy of the learned metric relative to the underlying true generative metric. As an application of these ideas, I will discuss work with collaborators in Educational Psychology that applies metric learning for perceptual feature detection in non-verbally mediated cognitive processes.

Feb 9, Adrienne Wood

Form follows function: Emotion expressions as adaptive social tools

Emotion expressions convey people’s feelings and behavioral intentions, and influence, in turn, the feelings and behaviors of perceivers. I take a social functional approach to the study of emotion expression, examining how the physical forms of emotion expression are flexible and can be adapted to accomplish specific social tasks. In this talk, I discuss two lines of research, the first of which applies a social functional lens to smiles and laughter. I present work suggesting that smiles and laughter vary in their physical form in order to achieve three distinct tasks of social living: rewarding others, signaling openness to affiliation, and negotiating social hierarchies. My approach, which generalizes to other categories of expressive behavior, accounts for the form and context of the occurrence of the expressions, as well as the nature of their influence on social partners. My second line of research examines how cultural and historical pressures influence emotional expressiveness. Cultures arising from the intersection of many other cultures, such as in the U.S., initially lacked a clear social structure, shared norms, and a common language. Recent work from my collaborators and myself suggests such cultures increase their reliance on emotion expressions, establishing a cultural norm of expressive clarity. I conclude by presenting plans to quantify individual differences in the tendency to synchronize with and accommodate to the emotion expressive style of a social partner, and relate those differences to people’s social network positions. Given the important social functions served by emotion expression, I suggest that the ability to use it flexibly is associated with long-term social integration.

Feb 16 Nils Ringe, Professor, Political Science

Speaking in Tongues: The Politics of Language and the Language of Politics in the European Union

Politics in the European Union primarily takes place between political actors who do not share a common language, yet this key feature of EU politics has not received much attention from political scientists. This project investigates if and how multilingualism affects political processes and outcomes. Its empirical backbone is a series of in-depth interviews with almost 100 EU policy-makers, administrators, translators, and interpreters, but it also involves at least two potential components where political science, linguistic, and computational approaches overlap. The first involves the analysis of oral legislative negotiations in the European Parliament, where non-native English speakers interact using English as their shared language, and in native-English speaking parliamentary settings (in Ireland, Scotland, and/or the UK), to determine if “EU English” differs syntactically and semantically from “regular” English. The expectation is that speech in the EP is simpler, more neutral, and more utilitarian. The second component involves the identification of languages spoken in EP committee meetings using computational methods, to determine the language choices members of the EP make.

Feb 23 Andreas Obersteiner

Does 1/4 look larger than 1/3? The natural number bias in comparing symbolic and nonsymbolic fractions

When people compare the numerical values of fractions, they are often more accurate and faster when the larger fraction has the larger natural number components (e.g., 2/5 > 1/5) than when it has the smaller components (e.g., 1/3 > 1/4). However, recent studies produced conflicting evidence of this “natural number bias” when the comparison problems were more complex (e.g., 25/36 vs. 19/24). Moreover, it is unclear whether the bias also occurs when fractions are presented visually as shaded parts of rectangles rather than as numerical symbols. I will first present data from a reaction time study in which university students compared symbolic fractions. The results suggest that the occurrence and strength of the bias depends on the specific type of comparison problems and on people’s ability to activate overall fraction magnitudes. I will then present preliminary data from an eye tracking study in which university students compared rectangular fraction visualizations. Participants’ eye movements suggest that the pure presence of countable parts encouraged them to use unnecessary counting strategies, although the number of countable parts did not bias their decisions. The results have implications for mathematics education, which I will discuss in the talk.

Mar 2 Nicole Beckage, University of Kansas

Title: Multiplex network optimization to capture attention to features

Abstract: How does attention to features and current context affect people’s search in mental and physical spaces? I will examine how procedures for optimally searching through “multiplex” networks — networks with multiple layers or types of relationships — capture human search and retrieval patterns. Prior work on semantic memory, people’s memory for facts and concepts, has primarily focused on modeling similarity judgments of pairs of words as distances between points in a high-dimensional space (e.g., LSA by Laudauer et al, 1998; Word2Vec by Mikolov et al. 2013). While these decisions seem to accurately account for human similarity in some contexts, it’s very difficult to interpret high dimensional spaces, making it hard to use such representations for scientific research. Further, it is difficult to adapt these spaces to a specific context or task. Instead, I define a series of individual feature networks to construct a multiplex network, where each network in the multiplex captures a “sense” or type of similarity between items. I then optimize the “influence” of each of these feature networks within the multiplex framework, using real world search behavior on a variety of tasks. These tasks include semantic memory search in a cognitive task and information search in Wikipeida. The resulting weighting of the multiplex can capture aspects of human attention and contextual information in these diverse tasks. I explore how this method can provide interpretability to multi-relational data in psychology and other domains by developing an optimization framework that considers not only the presence or absence of relationships but also the nature of the relationships. While I focus on applications of semantic memory, I discuss mathematical proofs and simulation experiments that apply more generally to optimization problems in the multiplex network literature.

Mar 9 Psychology visit day Data Blitz! (WID Orchard Room)

The UW-Madison Psychology department will be hosting its second annual data blitz for prospective graduate students in the Orchard Room of the WID. The data blitz will feature speakers from across the department presenting their research in an easily digestible format. Each talk will be 5 minutes long with an additional 2 minutes for questions at the end of each talk. All are welcome to attend. Below you will find a list of the speakers and the titles of their talks:

    • Rista Plate “Unsupervised learning shifts emotion category boundaries”
    • Ron Pomper “Familiar object salience affects novel word learning”
    • Elise Hopman “Measuring meaning alignment between different languages”
    • Anna Bartel “How diagrams influence students’ mental models of mathematical story problems”
    • Aaron Cochrane “Chronic and phasic interactions between video game playing and addiction”
    • Pierce Edmiston “Correlations between programming languages and beliefs about programming”
    • Chris Racey “Neural processing underlying color preference judgments”
    • Sofiya Hupalo “Corticotropin-releasing factor (CRF) modulation of frontostriatal circuit function”

Mar 16 Varun Jog, Assistant Professor, ECE

Title: Mathematical models for social learning

Abstract: Individuals in a society learn about the world and form opinions not only through their own experiences, but also through interactions with other members in the society. This is an incredibly complex process, and although it is difficult to describe it completely using simple mathematical models, valuable insights may be obtained through such a study. Such social learning models have turned out to be a rich source of problems for probabilists, statisticians, information theorists, and economists. In this talk, we survey different social learning models, describe the necessary mathematical tools to analyze such models, and give examples of results that one may prove through such an approach.

Mar 23 Josh Cisler, Assistant Professor, Department of Psychiatry

Title: Real-time fMRI neurofeedback using whole-brain classifiers with an adaptive implicit emotion regulation task: analytic considerations

Most fMRI neuroimaging studies manipulate a psychological or cognitive variable (e.g., happy versus neutral faces) and observe the manipulations impact on brain function (e.g., amygdala activity is greater for happy faces). As such, the causal inferences that can be drawn from these studies is the effect of cognition on brain function, and not the effect of brain function on cognition. Real-time fMRI refers to processing of fMRI data simultaneous with data acquisition, enabling feedback of current brain states to be presented back to the participant in (near) real-time, thus enabling the participant to use the feedback signals to modify brain states. We are conducting an experiment using real-time fMRI neurofeedback where the feedback signal consists of classifier output (hyperplane distances) from a SVM trained on all grey matter voxels in the brain. The feedback signal is embedded within a commonly used implicit emotion regulation task, such that the task becomes easier or harder depending on the participant’s brain state. This type of ‘closed loop’ design allows for testing whether manipulations of brain state (via feedback) have a measurable impact on cognitive function (task performance). The purpose of this presentation will be to present the experimental design and resulting data properties for the purpose of obtaining feedback and recommendations for understanding and analyzing the complex dynamical systems relations between the feedback signal, brain state, and task performance.

Mar 30 (no meeting, spring break)

Apr 6 Student Research Presentations

Mini talk 1: Ayon Sen, Computer Sciences

For Teaching Perceptual Fluency, Machines Beat Human Experts

In STEM domains, students are expected to acquire domain knowledge from visual representations. Such learning requires perceptual fluency: the ability to intuitively and rapidly see what concepts visuals show and to translate among multiple visuals. Instructional problems that enhance perceptual fluency are highly influenced by sequence effects. Thus far, we lack a principled approach for identifying a sequence of perceptual-fluency problems that promote robust learning. Here, we describe a novel educational data mining approach that uses machine learning to generate an optimal sequence of visuals for perceptual-fluency problems. In a human experiment realted to chemistry, we show that a machine-generated sequence outperforms both a random sequence and a sequence generated by a human domain expert. To our knowledge, our study is the first to show that an educational data mining approach can yield desirable difficulties for perceptual learning.

Mini talk 2: Evan Hernandez, Ara Vartanian, Computer Sciences

Block-based programming environments are popular in computer science education, but the click-and-drag style of these environments render them inaccessible by students with motor impairments. Vocal user interfaces (VUIs) offer a popular alternative to traditional keyboard and mouse interfaces. We design a VUI for Google Blockly in the traditional Turtle/LOGOS setting and discuss the relevant design choices. We then investigate augmentations to educational programming environments. In particular, we describe a method of program synthesis for completing the partial or incorrect programs of students, and ask how educational software may leverage program synthesis to enhance student learning.

Apr 13 (no meeting)

Apr 20

Rob Nowak: All of Machine Learning

Apr 27 (STARTING AT 4PM instead of 3:45pm). Tyler Krucas, The Wisconsin Gaming Alliance.

An Industry Perspective on Data in Game Design and Development

The Wisconsin game development industry offers a surprisingly comprehensive cross section of the types of individuals and teams that develop video games. This includes everything from studios that collaborate on AAA titles such as Call of Duty and Bioshock Infinite, to studios that work largely on mobile or free-to-play games, to studios that primarily work on educational games or games for impact. In all cases, data collection and analysis is an important tool in every step of the game development process. However, the scale of the data collected and its use can vary dramatically from developer to developer. In my talk, I will provide an overview of the the game development ecosystem in Wisconsin, as well as examples of the different types data collection and use practices found in the regional industry. Critically, I’ll frame this discussion in the context of possible links with the HAMLET group – in terms of possible sources of data to address fundamental questions surrounding human learning or behavior as well as possible collaborations.

Fall 2017 Schedule:

Sept 15, Virtual and Physical: Two Frames of Mind, Bilge Mutlu (CS)

In creating interactive technologies, virtual and physical embodiments are often seen as two sides of the same coin. They utilize similar core technologies for perception, planning, and interaction and engage people in similar ways. Thus, designers consider these embodiments to be broadly interchangeable and choice of embodiment to primarily depend on the practical demands of an application. In this talk, I will make the case that virtual and physical embodiments elicit fundamentally different frames of mind in the users of the technology and follow different metaphors for interaction. These differences elicit different expectations, different forms of engagement, and eventually different interaction outcomes. I will discuss the design implications of these differences, arguing for different domains of interaction serving as appropriate context for virtual and physical embodiments.

October 13, Learning semantic representations for text: analysis of recent word embedding methods, Yingyu Liang (CS)

Recent advances in natural language processing build upon the approach of embedding words as low dimensional vectors. The fundamental observation that empirically justifies this approach is that these vectors can capture semantic relations. A probabilistic model for generating text is proposed to mathematically explain this observation and existing popular embedding algorithms. It also reveals surprising connections to classical notions such as Pointwise Mutual Information in computational linguistics, and allows to design novel, simple, and practical algorithms for applications such as embedding sentences as vectors.

October 20, Vlogging about research, Martina Rau (Ed Psych)

October 27, LUCID faculty meeting, No large group meeting

November 3, A Discussion of Open Science Practices, Martha W. Alibali (Psych)

This HAMLET session will be a discussion of open-science practices, led by Martha Alibali. We will start with brief discussion of the “replication crisis” and “questionable research practices”. We will then discuss solutions, including better research practices, data sharing and preregistration. Please read at least some of the provided papers, and come prepared to ask questions and share your experiences.

Replication crisis paper

Questionable Research Practices (QRPs) and solutions paper paper

Data sharing paper http://deevybee.blogspot.co.uk/2014/05/data-sharing-exciting-but-scary.html

Preregistration paper pic https://www.psychologicalscience.org/observer/seven-selfish-reasons-for-preregistration

November 10, Systematic misperceptions of 3D motion explained by Bayesian inference, Bas Rokers (Psych)

Abstract: Over the years, a number of surprising, but seemingly unrelated errors in 3D motion perception have been reported. Given the relevance of accurate motion perception to our everyday life, it is important to understand the cause of these perceptual errors. We considered that these perceptual errors might arise as a natural consequence of estimating motion direction given sensory noise and the geometry of 3D viewing. We characterized the retinal motion signals produced by objects moving along arbitrary trajectories through three dimensions and developed a Bayesian model of perceptual inference. The model predicted a number of known errors, including a lateral bias in the perception of motion trajectories, and a dependency of this bias on stimulus contrast and viewing distance. The model also predicted a number of previously unknown errors, including a dependency of perceptual bias on eccentricity, and a surprising tendency to misreport approaching motion as receding and vice versa. We then used standard 3D displays as well as a virtual reality (VR) headsets to test these predictions in naturalistic settings, and established that people make the predicted errors. In sum, we developed a quantitative model of 3D motion perception and provided a parsimonious account for a range of systematic perceptual errors in naturalistic environments.

November 17, Total variation regression under highly correlated designs, Becca Willett (ECE)

Abstract: I will describe a general method for solving high-dimensional linear inverse problems with highly correlated variables. This problem arises regularly in applications like neural decoding from fMRI data, where we often have two orders of magnitude more brain voxels than independent scans. Our approach leverages a graph structure that represents connections among voxels in the brain. This graph can be estimated from side sources, such as diffusion-weighted MRI, or from fMRI data itself. We will explore the underlying models, computational methods, and initial empirical results. This is joint work with Yuan Li and Garvesh Raskutti.

November 24 Thanksgiving Holiday

December 1, Micro-(Shape-And-Motion)-Scopes, Mohit Gupta (CS)

Imagine a drone looking for a safe landing site in a dense forest, or a social robot trying to determine the emotional state of a person by measuring her micro-saccade movements and skin-tremors due to pulse beats, or a surgical robot performing micro-surgery inside the body. In these applications, it is critical to resolve fine geometric details, such as tree twigs; to recover micro-motion due to biometric signals; and the precise motion of a robotic arm. Such precision is more than an order-of-magnitude beyond the capabilities of traditional vision techniques. I will talk about our recent work on designing extreme (micro) resolution 3D shape and motion sensors using unconventional, but low-cost optics, and computational techniques. These methods can measure highly subtle motions (< 10 microns), and highly detailed 3D geometry (<100 microns). These sensors can potentially detect a persons pulse or micro-saccade movements, and resolve fine geometric details such as a facial features, from a long distance.

December 8, Influence maximization in stochastic and adversarial settings, Po-Ling Loh (ECE)

We consider the problem of influence maximization in fixed networks, for both stochastic and adversarial contagion models. Such models may be used to model infection spreads in epidemiology, as well as the diffusion of information in viral marketing. In the stochastic setting, nodes are infected in waves according to linear threshold or independent cascade models. We establish upper and lower bounds for the influence of a subset of nodes in the network, where the influence is defined as the expected number of infected nodes at the conclusion of the epidemic. We quantify the gap between our upper and lower bounds in the case of the linear threshold model and illustrate the gains of our upper bounds for independent cascade models in relation to existing results. In the adversarial setting, an adversary is allowed to specify the edges through which contagion may spread, and the player chooses sets of nodes to infect in successive rounds. Our main result is to establish upper and lower bounds on the regret for possibly stochastic strategies of the adversary and player. This is joint work with Justin Khim (UPenn) and Varun Jog (UW-Madison).

For other events check out our calendar: Seminar and Events  This content and updates can be found: HAMLET

HAMLET Archives

Fall 2015 archive
Fall 2012 archive
Fall 2011 archive
Spring 2011 archive
Fall 2010 archive
Fall 2009 archive
Spring 2009 archive
Fall 2008 archive