learning.understanding.cognition.intelligence.data science


HAMLET  (Human, Animal, and Machine Learning: Experiment and Theory)

HAMLET is an interdisciplinary proseminar series that started in 2008. The goal is to provide behavioral and computational graduate students with a common grounding in the learning sciences. Guest speakers give a talk each week, followed by discussions. It has been an outlet for fresh research results from various projects. Participants are typically from Computer Sciences, ECE, Psychology, Educational Psychology as well as other parts of the campus. Multiple federal research grants and publications at machine learning and cognitive psychology venues have resulted from the interactions of HAMLET participants.

Meetings: Fridays 3:45 p.m. – 5 p.m. Berkowitz room (338 Psychology Building)

Subscribe/Unsubscribe: HAMLET mailing list

This semester we are trying a new format for HAMLET. Speakers pursuing research at the intersection of computation and human behavior will give a short (10-15 minute) introduction/outline/description of a current research project or problem, with the aim of sparking collaborative discussion in the group about the project.

How would people from your field approach it?
Are there existing tools for solving the problem?
What is already known about the domain?
What kind of data are involved?
How does the research connect to real-world issues?

Brainstorm sessions will be interleaved with standard talks on more mature work throughout the semester.

Fall 2020 Schedule:

HAMLET this semester will be virtual, make sure that you are subscribed to the HAMLET list in order to get the link to the meeting.

If there is any silver lining to our current national struggles, it is that it is easier than ever to remotely bring in speakers from across the globe.

10/2 Paul Kantor with Vicki Bier, Gary Lupyan and Jerry Zhu

Paul Kantor will be discussing a recently funded in-development project, comparing rule-learning by humans and machines.

Title: Human and Machine Learning: Searching for Differences

Abstract: We are using “rule finding games” to look for anomalous pairs of rule classes. A pair is anomalous if the class that is more difficult for humans is easier for machines.  The study will use a “table clearing” game, in which figures with varying shape, color, and position are displayed on a screen with four “buckets” in the corners. The rules determine the order in which the objects are to be placed into buckets, and the target placement for each object. Examples might be “circles in bucket A; all others anywhere” or “put each object in its closest bucket, farthest first, and decreasing to the one whose closest bucket is nearest.” The space of possible rules is enormous, permitting stochastic rules, history-dependent rules, and rules depending on any aspect of the object that is visible to humans.  I will talk about how we got started, and a bit about where we hope to arrive, and even less about what we already know.

10/9 Sebastian Musslick: On the artional bounds of cognitive control

Sebastian Musslick, a scientist in Jonathan Cohen’s lab at the Princeton Neuroscience Institute. Sebastian’s work involves mathematical and simulation analyses of neural network models to understand a central aspect of human cognition: why can’t we do many different cognitive tasks in parallel, and how is it that we sometimes can? Put differently: Why can we walk and chew gum, but can’t text and drive?

Title: On the Rational Bounds of Cognitive Control

Abstract: One of the most remarkable features of the human brain is its ability to adapt behavior in a changing world, e.g. avoiding to touch one’s face during elevated risk of viral infection. Mechanisms underlying this function are summarized under the term cognitive control, and are engaged across all the higher mental faculties that distinguish us as
humans, including language, reasoning and problem solving. Yet, humans are strikingly limited in how many control-demanding tasks they can perform at the same time (e.g. reading a document while listening to a colleague). These limitations became a fundamental premise for most general theories of human cognition. However, none of these theories
provides a rational explanation for why neural systems, such as the human brain, would be limited in the number of control-demanding tasks it can perform in parallel. In this talk, I argue that such constraints on multitasking capability may result from a rational adaptation to a fundamental computational dilemma in neural architectures. Using neural network simulations and behavioral experiments I demonstrate that neural architectures are subject to a tradeoff between learning efficacy, that is promoted through the use of shared task representations, on the one hand, and multitasking capability, that is achieved through the separation of task representations, on the other hand. These results suggest that the limited capability to execute multiple tasks in parallel may reflect a preference of the neural system to learn tasks more quickly. I conclude by suggesting that the tradeoff between shared and separated representations, and its interaction with learning, represent a fundamental principle of adaptive network architectures that underlies and shapes all domains of psychological function, from perception and inference to task execution, and extends equally to artificial systems.

10/23 Ramya Vinayak: Crowdsourced Clustering: Query design and Robust Algorithms

Ramya Vinayak has been doing some exciting work using crowdsourcing of human judgments combined with insights from adaptive sampling to efficiently estimate human-perceived structure in data. Ramya is a new faculty member in ECE
and her work dovetails very nicely with work on the NEXT system and other efforts from both computational and behavioral sciences to develop contemporary techniques for quickly finding structure in human-generated

Title: Crowdsourced Clustering: Query design and Robust Algorithms

Abstract: Crowdsourcing is a popular way of collecting labels for unlabeled data to create training data for supervised learning tasks. Collecting labels via crowdsourcing is a noisy process as the workers are usually not domain experts. This makes it difficult to obtain high quality granular data. For example, the workers can easily say if an animal is a dog or a
cat but find it difficult to identify the breed of a dog. To overcome this limitation, we propose clustering answers to easier comparison queries. We introduce triangle query (clustering three images per query) for this crowdsourced clustering task and compare it with edge queries (clustering two images per query).  Via experiments on real datasets collected on Amazon Mechanical Turk, we show that for a fixed budget, our design can double the amount of data collected while simultaneously reducing noise. Another important issue in crowdsourcing is that of budget constraint which puts a limitation on the number of queries one can make. Therefore, we need robust algorithms that can perform clustering on partially observed, noisy data. We analyze convex algorithms based on low-rank plus sparse decomposition of matrices and show that they can reliably find clusters in the presence of noise, outliers and missing data both in theory and in practice.

10/30 Patrick Shafto: A story of (children’s) learning, love (of math), and (minimizing) loss

Pat directs the Cognition and Data Science lab at Rutgers, and is Rutgers Term Chair in Data Science. He is also one of the few people Tim knows who moved from an faculty appointment in a Psychology Department to an appointment in a
Math/CS Department–so right up HAMLET’s alley.

Pat’s work focuses on using Bayesian models and other computational approaches in aid of understanding human learning and teaching, especially in educational contexts.

Title: A story of (children’s) learning, love (of math), and (minimizing) loss

Starting from the goal of having more rigorous theories of human and machine learning, I will offer a light critique of the state of the art in human and machine learning. I will then introduce a new, rather old, mathematical approach that connects child development, machine learning, and pure mathematics. The technical parts are based on a new paper “A mathematical theory of cooperative communication” to appear in NeurIPS this year.

11/6 Siddharth Suresh & Emily Ward

Title: Visual ensemble representations emerge in Deep Neural Networks trained for object recognition

Becoming aware of the world seems simple and straightforward — but we frequently see the world in a way that differs from how it actually is. We may fail to see a car turning right in front of our eyes, while still feeling like what we see is extremely rich and detailed. How can we perceive so much richness when we are susceptible to such failures of awareness? One possibility is that we perceive ensemble properties of scenes, without being aware of the many individual features. Humans are able to quickly pool information from across many individual objects to perceive ensemble properties, like the average size of objects, the color diversity of objects, or even the gender ratio in a group of faces. Are such ensemble properties also present in artificial visual systems, such as Deep Convolutional Neural Networks? And if so, might these networks also be useful for understanding visual awareness, in addition to their utility in understanding object recognition? Here, we determined whether the ensemble property of color diversity emerges in a network pre-trained to recognize natural objects (using ImageNet). We presented the network with new images that were completely different from its training set: images of letter arrays containing four colored consonants on a black background. The color of each letter within an array was drawn either from a broad sample of colors (high color diversity) or a randomly selected range of adjacent colors (low color diversity). We tested whether a ResNet50 neural network could distinguish high vs. low color diversity arrays by using the activations from different layers as input to a linear classifier (SVM). We found that the network was highly accurate at identifying both color and color diversity. This demonstrates the presence of ensemble representations for the first time in deep neural networks, and also suggests that ensemble perception of multiple objects can emerge even in a system only trained to recognize individual objects. We will discuss how the emergence of ensemble representations in neural networks may make such networks suitable models for failures of visual awareness.

11/13 Gary Lupyan: Can telepathy work?

Science fiction has long used telepathy as a method of efficient communication. The common trope is an advanced alien species communicating with humans telepathically without, of course, sharing a language. Several companies, including Elon Musk’s Neuralink project, aim to make this science fiction a reality. A common assumption in both fictional and aspirational narratives of telepathy is that natural language is slow, imprecise, and mendacious. By bypassing language, a direct mind-to-mind connection is thought to make communication faster, more precise, and truthful.
But could it work? Rather than focusing on the technological aspects of the question, I am going to focus on the psychological. All successful communication requires the signaler and the recipient to agree on a set of mutually understood signals. Linguistic communication fulfills this goal through a learned, shared vocabulary. For telepathy (at least the sci-fi kind) to work requires a shared nonlinguistic vocabulary — a mentalese — that can be used for communication. Do we, in fact, have such a vocabulary? How can we find out?

11/20 Jackie Fulvio: Introducing a new Gender Citation Balance Index web tool for academics

It has recently been reported that in five broad-scope neuroscience journals, citation rates for papers first- and/or last-authored by women are lower than would be expected if gender was not a factor in citation choices (Dworkin et al., 2020). Given the important implications that such underrepresentation may have on the careers of women researchers, it is important to determine whether this same trend is true in subdisciplines of the field, where interventions might be more targeted. We extended the analyses carried out by Dworkin et al. (2020) to citation patterns in the Journal of Cognitive Neuroscience (JoCN) and determined that the reference sections of papers published in JoCN during the last decade or so included women-led papers at a level considerably lower than the expected base rate given the journal’s authorship. Furthermore, this pattern of citation imbalances has been characteristic of the reference sections in all author-gender groups (i.e., “MM,” “MW,” “WM,” and “WW”), thereby implicating systemic factors. These results contribute to the growing body of evidence that intentional action is needed to address inequities in the way that we carry out and communicate our science. To that end, I developed a web tool that went “live” in late October, 2020, that accepts an author’s reference list and returns the categorical gender breakdown. I will introduce this tool with a deep dive into how it works. I’ll then discuss how we had envisioned this tool would be used in cognitive neuroscience and give a brief report about useability and feedback, successes and failures, to date. Finally, I’ll discuss possible uses and extensions of the tool beyond its initial application in cognitive neuroscience.

11/27 Thanksgiving

12/4 James Evans


Fall 2019 Schedule:

HAMLET this semester will have a mini-series on neural networks.

9/27 Gary Lupyan lead a group discussion on what does it mean to understand a neural network?

What does it mean to understand what a neural network is doing? How do people in different disciplines–cognition, neuroscience, machine learning–think about or answer this question? To orient the discussion, please have a look at the following recent paper by Tim Lillicrap (machine learning scientist at Deep Mind) and Konrad Kording (computational neuroscientist at Penn):


We are very interested to know whether different disciplines even agree on what “interpretability” means, and whether there is useful cross-disciplinary progress to be made.

10/4 Yingyu Liang

Title: Robust Attribution Regularization
Abstract: An emerging problem in trustworthy machine learning is to train models that produce robust interpretations for their predictions. We take a step towards solving this problem through the lens of axiomatic attribution of neural networks. Our theory is grounded in the recent work, Integrated Gradients (IG), in axiomatically attributing a neural network’s output change to its input change. We propose training objectives in classic robust optimization models to achieve robust IG attributions. Our objectives give principled generalizations of previous objectives designed for robust predictions, and they naturally degenerate to classic soft-margin training for one-layer neural networks. We also generalize previous theory and prove that the objectives for different robust optimization models are closely related. Experiments demonstrate the effectiveness of our method, and also point to intriguing problems which hint at the need for better optimization techniques or better neural network architectures for robust attribution training.

This is joint work with Jiefeng Chen, Xi Wu, Vaibhav Rastogi, and Somesh Jha.

10/11 Rahul Parhi: Neural Networks Learn Splines

Where: State Room (4th floor) in Memorial Union

We develop a general framework based on splines to understand the interpolation properties of overparameterized neural networks. We prove that minimum “norm” two-layer neural networks (with appropriately chosen activation functions) that interpolate scattered data are minimal knot splines. Our results follow from understanding key relationships between notions of neural network “norms”, linear operators, and continuous-domain linear inverse problems.

10/18 Larissa Albantakis: Causation vs prediction: Understanding *why* an agent did what it did

As part of our mini-series on neural networks, Larissa Albantakis from the Wisconsin Institute for Sleep and Consciousness will be speaking on the topic of causation vs prediction in understanding the
behavior of artificial agents. Larissa has training in physics and
computational neuroscience, and has developed computational and
neural-network-based approaches to difficult problems in cognition and
neuroscience, including contributions to the integrated-information approach to consciousness and to theories of human decision-making.

When an agent interacts with its environment, we are often interested in  why the agent performed a particular action. Due to recent advances in the field of artificial intelligence, there is a growing realization that the ability to predict what a system is going to do does not equal understanding how or why it behaves in a certain way, not even in hindsight. This is demonstrated particularly well by recent computational studies involving simulated artificial agents with minimal cognitive architectures, whose behavior can easily be predicted. Yet, understanding what caused the agent to perform a particular action typically requires extensive additional analysis and cannot be addressed in purely reductionist or holistic terms. Finally, the same action can be performed for different reasons and with different levels of autonomy. How can we distinguish reflexes from autonomous actions in mechanistic terms?

10/25 Claudia Solis-Lemus: Deep learning methods to predict antibiotic-resistance in microbial data

Abstract: One of the golden standards of precision medicine is the accurate prediction of an individual’s antibiotic response based on host’s as well as bacteria’s genomic data, especially for severely infectious pathogens like Staphylococcus aureus and Pseudomonas aeruginosa. I will present the recent work on deep learning techniques to predict antibiotic-resistant traits on microbial genomic data. I will focus mostly on the challenges that users normally face when trying to fit deep learning models from the choice of architecture and other parameters to small sample sizes with missing and sparse data.

11/1 Mike Zinn

Where: Union South, Landmark Room

Title: Toward a Robotically-Guided Diagnostic and Rehabilitation Paradigm for the Treatment of Motor Skill Deficiencies in Children with Autism Spectrum Disorder

Many children with autism spectrum disorder (ASD) struggle with motor impairments and these motor impairments have been found to predict difficulties with independent living skills and the severity of core social communication impairments. While many of us take for granted how intertwined our motor skills are with our social and cognitive abilities, motor difficulties in autism may drastically limit a child’s ability to explore the world, competently interact with others, and participate in activities with peers. As such, motor skill therapy can offer a unique and under-utilized therapeutic approach to improve the independent living skills and social skills of children with autism.

One challenge in training motor skills in ASD, is that individuals with autism have a unique motor learning style that necessitates utmost simplicity and consistency in presentation, which is difficult for a human therapist to enact with fidelity. In addition, diagnosis and assessment is commonly performed using motor skill metrics where the assessment relies on the expertise and, to some degree, qualitative judgment of the therapist. As such current approaches for assessment of motor skills are time consuming, prone to error, and can result in widely varying diagnosis depending on the skill and experience of the assessing clinician.

Is there a more direct, quantitative approach for both assessment and treatment?   When considering current assessment methods, they focus on measuring the expression of the impairment, in terms of specific task skills, will little to no consideration of underlying physical and/or neurological characteristics. What if these could be measured, using robotic rehabilitation to excite and measure the underlying dynamic system characteristics while also providing the consistency in presentation required when working with children with ASD.

Could accurate measurements be used to develop mathematical models and dynamic system descriptions of the underlying source of the motor skill impairment?  Would these mathematical models relate to observed motor skill deficiencies and would this description provide a window into the mechanisms of the motor impairments?  Would it provide a possible avenue through which to target and assess treatment approaches by seeking to modify the structure or parameters of the dynamic system models associated with motor skill deficiency?  To answer these questions, we need to understand the relationship and connections between the measurements of physical interaction between human patient and robot, assessment of motor skill proficiency, coupled with ASD diagnostic assessment.

The talk will present preliminary human-study data and discuss the host of challenges that remain. The work presented here is from an on-going collaboration between Brittany Travers (Kinesiology), Andrea Mason (Kinesiology), and Mike Zinn (Mechanical Engineering)

11/8 Peter Adamczyk and Alexander Dawson-Elli
Title: The Challenge of Eliciting and Modeling Motor Learning

In addition to explicit knowledge, the brain and other neural structures must learn and apply knowledge of how to control the body to move and accomplish tasks. This Motor Learning, and its short-term cousin Motor Adaptation, have distinct characteristics, such as exponential convergence over time, specific adaptation patterns to different conditions, and circumstances in which it can and cannot be accomplished. Motor Learning has been used as a guide for developing rehabilitation strategies for persons with neuromotor injury such as stroke, but with limited success. One of the challenges is developing conceptual and computational models of the state of the neural motor control system and using these to predict outcomes and adapt rehabilitative tasks. This seminar will provide a primer on principles of motor learning, and present some challenges in modeling that we hope HAMLET participants can help address.

11/15 Bilge Mutlu & David Porfirio

Title: Computational Tools to Support the Design of Artificial Agents

Abstract: Artificial characters—from chatbots to social robots—are expected to follow human norms and mechanisms of communication, and designers of these characters are expected to encode such norms and mechanisms into their systems. How can designers formalize this information into computational representations? How can they ensure that systems will follow these specifications in their interactions with people? How can these systems customize and adapt their behaviors to the needs, expectations, and preferences of their users? In this talk, we will give an overview of a new research program that bridges interaction design and formal methods in the context of designing artificial agents toward building design support tools. We will present our vision for such tools, illustrate our approach with three tools we have built, and conclude with a discussion of our future work.

11/22 Yin Li

Title: Towards Computational Imaging of Daily Activities–From Body Movements to Human-Object Interactions

Abstract: Major progress has been made in visual sensing and computer vision towards human body pose estimation. This has enabled novel means of measuring our purposive body movements in the context of daily activities using minimally intrusive cameras. The ability to measure body motion in turn provides valuable insights to assess our physical activities. In this talk, I will describe our ongoing efforts on sensing and modeling human body movements for understanding human-object interactions. First, I will present our work on future interaction anticipation from a first person video captured by a body-worn camera. We demonstrate that modeling of the camera wearer’s hand trajectory leads to more accurate prediction of actions in the near future. Second, I will discuss our work on estimating the weight of an object that a user is lifting from a third person video. The key hypothesis is that the lifting load is characterized by distinct patterns of a user’s joint trajectory. I will describe our pilot study and preliminary results. Finally, I will briefly introduce our work on developing wrist-worn visual sensor for continuous 3D finger pose tracking. Putting things together, I will discuss the new opportunities for imaging our daily activities.

11/29 Thanksgiving – no HAMLET

12/6 Michael Gleicher

Title: Dumb Robots for Smart People

Abstract: Even with advances in intelligence and learning, there are still places where people need to control robots. Examples include tele-operation for automation resistant tasks and providing demonstrations for robot learning. In such situations, direct interfaces, where a person’s movements are mapped to the robot, can be advantageous. In this talk, I will describe our recent efforts to create better direct control interfaces. First, I will discuss our work on arm-scale tele-operation systems based on mimicry, the idea of making the system feel as if it is mimicking a users’ movement. I will describe how we address the basic challenges of mapping movements to robots which allows us to see the benefits of direct control. I will then describe extensions that use elements of shared control to address challenges in watching what the robot is doing as well as performing two handed tasks. Second, I will discuss our work on developing input devices that permit people to demonstrate tasks to robots in a natural, yet constrained, manner. I will show how our ability to better instrument demonstrations provides information that allows us to infer the mechanical properties of the actions, which allows for more informed use of the demonstrations. With both projects, I will discuss the opportunities for combining direct control with intelligence to create even better performance.

Spring 2019 Schedule:

2/8 John Curtin, Psychology

Brainstorm: Digital phenotyping with (somewhat) big data for harm reduction in alcohol and other drug use disorders

John has been applying machine learning methods to Facebook and geo-tracking data to understand and predict patterns of relapse in
people recovering from addiction. He will briefly characterize issues that motivate the research, the kinds of data his group is working with,
some of the work they have already done and some of the challenges they are currently facing. The discussion will focus on generating ideas for moving the project forward.

2/15: Data blitz: Current research in cognition, perception, and cognitive neuroscience (special event at the WID)

2/22: Paula Niedenthal, Psychology

Brainstorming connections between emotion, cognition and datascience

Paula will initiate a session concerning ways that scientists in machine learning/data science might collaborate with scientists working to understand human affective cognition. Read about some of her recent work here: Science Daily

3/1: Dimitris Papailiopoulos (ECE, WID)  LOCATION CHANGE: Union South (look at the TTIU for the room)

Dimitris Papailioupolos will spark a discussion assessing whether principles of human cognition can guide us toward more robust machine learning.

3/8 Alyssa Adams, VEDA Data Solutions

Emergence, evolution and video games

Alyssa Adams, a data scientist working at Madison startup VEDA Data Solutions. Alyssa’s work focuses on understanding what makes living systems different from non-living systems, and in particular the mechanisms that underlie emergence in social and biological systems. Toward these questions she applies computational approaches to data generated by people interacting in video games–so if you like emergence, math, human behavior and video games, you should really come. Alyssa is also a great person to talk to if you are interested in how data science is being applied in the wild, or what it is like to move from an academic to an industrial career path.

3/22  Spring Break

3/29 Karen Schloss, Psychology & WID & Laurent Lessard, WID/ECE

Using machines to learn how humans map colors to concepts

Karen Schloss and Laurent Lessard with lead a brainstorming session focusing on the use of machine learning to understand a central aspect of human cognition: what explains the conceptual inferences and associations that people draw from simple visual features, such as colors? Karen and Laurent will briefly set up the problem, describing some interesting phenomena and prior work, then will lead an interactive discussion on ways that machine learning might be developed or applied to solve key questions.

4/5 Jon Willits, Psychology at University of Illinois

A Two-Process Model of Semantic Knowledge

Jon’s work has connected large-corpus NLP-based approaches to word meaning with cognitive theories of language. He will present new work on this topic, suggesting a two-stage model of semantic processing with implications for both human language and the design of artificial knowledge systems.

Abstract: Computational models of semantic memory, and machine learning approaches to knowledge-based systems, have traditionally proposed that there is a single semantic memory – one set of factual knowledge that is
used for all activities. There are a number of problems (both theoretical and empirical) with this view. In this talk, I will propose an alternate, two-process model. Process 1 consists of a general, passive, statistical-learning system that acquires knowledge of general, context-insensitive associations and similarities in the environment.
Process 2 consists of task- or goal-specific subsystems, where stored semantic representations are changed in ways that optimize their use for specific situations. I will show how this model addresses a number of thorny theoretical and empirical issues in the study of semantic memory and building artificial knowledge systems.

4/12 Ari Rosenberg, Neuroscience

Inferring whole brain connectivity

Knowing the interconnections between brain regions is important to understand how neural functions are implemented by distributed networks, and how different networks share information. Diffusion tensor imaging (DTI) provides a technique for estimating anatomical pathways within the brain. Our preliminary attempts to create connectivity matrices for macaque cortex using DTI have proven promising. First, we find a surprisingly strong ability to predict functional connectivity data from other labs. Second, we find evidence of pathways hypothesized to exist based on electrophysiological data. A potential point of discussion is whether graph theory provides appropriate techniques for quantifying brain networks using DTI data.

4/19 Greg Zelinsky, Psychology at Stony Brook University

Predicting goal-directed attention control: A tale of two brain-inspired computational models

The ability to control the allocation of attention underlies all visuospatial goal-directed behavior; there would be no sense in having a goal if there was no way to use it to select visual inputs. He will summarize two recent efforts to computationally model this core mechanism of cognitive control. These models are similar in that both are image computable and that the design of each is inspired by mechanisms used by the attention-control system in the primate brain. But the models also differ in their type of brain inspiration and in their neurocomputational approach. One model adopts a computationally narrow approach but is closely parametrized by known neural constraints (MASC, a Model of Attention in the Superior Colliculus). The other model adopts a more powerful deep neural network approach but is less directly brain inspired (Deep-BCN, a deep network implementation of the widely accepted biased-competition theory of attention control). Together they are intended to span a continuum of brain-inspired model design, ranging from relatively local neural circuits at one end to the broader network of brain areas comprising the primate attention control system at the other.

4/26 Glenn Fung Moo, a machine learning scientist at American Family Insurance

Non-IID Model Learning via Graph Neural Networks

Most of the more commonly used machine learning methods assume that training and test data comes from independent and identically distributed (IID) random variables. However, this assumption is strongly violated when the data points available to learn/test the model are highly interdependent. Two examples of this scenario are: when the data exhibits temporal or spatial correlations and when the task is to learn relative characteristics between data points (e.g., ranking or attribute learning). In this talk we show how emerging ideas in graph neural
networks can yield a solution to various problems that broadly fall under this characteristics. More specifically, we show interesting results for the relative attribute learning problem from images. This setting, naturally benefits from exploiting the graph of dependencies among the different relative attributes of images, especially when only
partial ordering is provided at training time. Our experiments show that this simple framework is effective in achieving competitive accuracy with specialized methods for both relative attribute learning and binary attribute prediction.

5/3 Sarah Sant’Ana, Psychology



Fall 2018 Schedule:

Sep 14, Tim Rogers, Psychology

Investigating the covariance structure of widely-held false beliefs

ABSTRACT: It has become increasingly clear that our society is suffering a crisis of false belief. Across a variety of domains–including science, health, economics, politics, history, and current events–there often exists vociferous disagreement about what ought to be matters of fact. I’d like to know whether cognitive science and machine learning together can help us to understand why this is happening. In prior work we have studied the formation of false beliefs in small well-controlled lab studies with no real-world stakes. In this talk I will describe some preliminary work looking at real false beliefs occurring in the wild. In collaboration with a summer PREP student and colleagues in Psychology and in the School of Journalism, we generated a long list of incorrect but putatively widely-held beliefs spanning many different knowledge domains. Using Amazon Mechanical Turk, we asked people about the degree
to which they endorsed or rejected the belief. We also collected a variety of data about each respondent, including sociodemographic data, political leanings, media trust, IQ, and so on. The aim was to measure how susceptibility to false beliefs pattern across individuals, and to understand what properties of individuals predict susceptibility to which kinds of false beliefs. Our early results provide some provocative evidence contradicting some straight-forward hypotheses about false belief formation, but there is a lot more work to be done. We would like to connect with others interested in these problems from a machine-learning-and-social-media perspective, and in particular would like to consider ways of using social network analytics to assess the degree to which media consumption may be causing the patterns we observe.

Sep 21, LUCID faculty meeting – no HAMLET

Sep 28, no HAMLET

Oct 5, Jay Patel, Educational Psychology and Ayon Sen, Computer Science

A Novel Application of Machine Teaching to Perceptual Fluency Training, a collaboration between Ed Psych and CS

W will hear from Jay Patel and Ayon Sen, who will present new work applying neural networks and machine teaching to undergraduate chemistry learning—something for everyone!


In STEM domains, students are expected to acquire domain knowledge from visual representations that they may not yet be able to
interpret. Such learning requires perceptual fluency: the ability to intuitively and rapidly see which concepts visuals show and to translate among multiple visuals. Instructional problems that engage students in nonverbal, implicit learning processes enhance perceptual fluency. Such processes are highly influenced by sequence effects. Thus far, we lack a principled approach for identifying a sequence of perceptual-fluency problems that promote robust learning. Here, we describe a novel educational data mining approach that uses machine learning to generate an optimal sequence of visuals for perceptual-fluency problems. In a human experiment, we show that a machine-generated sequence outperforms both a random sequence and a sequence generated by a human domain
expert. Interestingly, the machine-generated sequence resulted in significantly lower accuracy during training, but higher posttest
accuracy. This suggests that the machine-generated sequence induced desirable difficulties. To our knowledge, our study is the first to show that an educational data mining approach can induce desirable difficulties for perceptual learning.

Oct 12, Tim Rogers, Psychology

Data science and brain imaging at UW Madison

Over the summer Tim had conversations with several groups about their work applying machine learning or other data science tools to brain imaging data. These activities are part of a broader trend across the country, with collaborations between neuroscientists, cognitive scientists, and computer scientists driving very rapid innovations in techniques for finding signal in neural data. Many of us are interested in connecting disparate efforts across campus to develop a general knowledge base, community of support, and new collaborations in this field. There is also interest in understanding how work conducted here relates to other high-profile work emerging from other labs.

Tim will present a general overview of the variety of new methods that I am aware of and some of the astonishing results they have uncovered. The overview will be targeted at both cognitive scientists/neuroscientists and members of the machine learning
community. I will also highlight some of the opportunities for new work and collaboration that I am aware of, and hope to initiate a discussion about what we can do to promote these efforts at UW.

Join the new email list to facilitate discussion, collaboration, and information-sharing on these issues.  big-lucid@lists.wisc.edu  (BIG = “Brain-imaging Interest Group”; LUCID is the broader community interested in learning, understanding, cognition, intelligence and data-science)

Oct 19, LUCID faculty meeting – no HAMLET

Oct 26, no HAMLET

Nov 2, Jerry Zhu, Computer Science

Discuusion: Can Machine Teaching Influence How Humans Do Regression?

Jerry is interested in discussing with cognitive scientists possible relationships between a hot topic in machine learning–data
poisoning, and defences against data poisoning–and approaches to human learning and teaching. Can central concepts from this area of study in machine learning help or hinder human learning?

How do people make numerical predictions (e.g. weight of a person) from observed features (height, fitness, gender, age, etc.)?  We may assume that they learn this from some sort of regression in their head, applied to training data they receive.  Can machine teaching help them learn better predictions by being careful about the training examples we give them?   Or to mess them up intentionally?  This is equivalent to ‘training data poisoning attacks’ in adversarial machine learning.   I will explain how this is done with control theory under highly simplified, likely incorrect assumptions on human learning.  This is where we need the cognitive science audience to participate in the discussion, and together we will explore the research opportunities on this topic.

Nov 9, Data Blitz

75 minutes, 9 talks, a sampling of current research at UW connecting data science and human behavior

This HAMLET will feature a cross-disciplinary data-blitz. We will hear a series of 5-minute talks on topics at the intersection of
machine learning, data science, and human behavior. This is a great opportunity to hear about some of the projects happening on campus now that you may want to get involved with.

Can neural networks help you learn chemistry? How do Trumpian false beliefs differ from other partisan false beliefs? These questions any many more will be addressed in this data blitz!

Nov 16, Gary Lupen

How translateable are languages? Quantifying semantic alignment of natural languages.

Do all languages convey semantic knowledge in the same way? If language  simply mirrors the structure of the world, the answer should be a qualified “yes”. If, however, languages impose structure as much as reflecting it, then even ostensibly the “same” word in different
languages may mean quite different things. We provide a first pass at a large-scale quantification of cross-linguistic semantic alignment of approximately 1000 meanings in 70+ languages. We find that the translation equivalents in some domains (e.g., Time, Quantity, and Kinship) exhibit high alignment across languages while the structure of other domains (e.g., Politics, Food, Emotions, and Animals) exhibits substantial cross-linguistic variability. Our measure of semantic alignment correlates with phylogenetic relationships between languages and with cultural distances between societies speaking the languages, suggesting a rich co-adaptation of language and culture even in domains of experience that appear most constrained by the natural world.

Nov 23, Thanksgiving – no HAMLET

Nov 30, Emily Ward, Psychology

Dec 7, Priya Kalra, Educational Psychology





Spring 2018 Schedule:

Feb 2, Blake Mason

Title: Low-Dimensional Metric Learning with Application to Perceptual Feature Selection

Abstract: I will discuss recent work investigating the theoretical foundations of metric learning, focused on four key topics: 1) how to learn a general low-dimensional (low-rank) metrics as well as sparse metrics; 2) upper and lower (minimax) bounds on the generalization error; 3) how to quantify the sample complexity of metric learning in terms of the dimension of the feature space and the dimension/rank of the underlying metric; 4) the accuracy of the learned metric relative to the underlying true generative metric. As an application of these ideas, I will discuss work with collaborators in Educational Psychology that applies metric learning for perceptual feature detection in non-verbally mediated cognitive processes.

Feb 9, Adrienne Wood

Form follows function: Emotion expressions as adaptive social tools

Emotion expressions convey people’s feelings and behavioral intentions, and influence, in turn, the feelings and behaviors of perceivers. I take a social functional approach to the study of emotion expression, examining how the physical forms of emotion expression are flexible and can be adapted to accomplish specific social tasks. In this talk, I discuss two lines of research, the first of which applies a social functional lens to smiles and laughter. I present work suggesting that smiles and laughter vary in their physical form in order to achieve three distinct tasks of social living: rewarding others, signaling openness to affiliation, and negotiating social hierarchies. My approach, which generalizes to other categories of expressive behavior, accounts for the form and context of the occurrence of the expressions, as well as the nature of their influence on social partners. My second line of research examines how cultural and historical pressures influence emotional expressiveness. Cultures arising from the intersection of many other cultures, such as in the U.S., initially lacked a clear social structure, shared norms, and a common language. Recent work from my collaborators and myself suggests such cultures increase their reliance on emotion expressions, establishing a cultural norm of expressive clarity. I conclude by presenting plans to quantify individual differences in the tendency to synchronize with and accommodate to the emotion expressive style of a social partner, and relate those differences to people’s social network positions. Given the important social functions served by emotion expression, I suggest that the ability to use it flexibly is associated with long-term social integration.

Feb 16 Nils Ringe, Professor, Political Science

Speaking in Tongues: The Politics of Language and the Language of Politics in the European Union

Politics in the European Union primarily takes place between political actors who do not share a common language, yet this key feature of EU politics has not received much attention from political scientists. This project investigates if and how multilingualism affects political processes and outcomes. Its empirical backbone is a series of in-depth interviews with almost 100 EU policy-makers, administrators, translators, and interpreters, but it also involves at least two potential components where political science, linguistic, and computational approaches overlap. The first involves the analysis of oral legislative negotiations in the European Parliament, where non-native English speakers interact using English as their shared language, and in native-English speaking parliamentary settings (in Ireland, Scotland, and/or the UK), to determine if “EU English” differs syntactically and semantically from “regular” English. The expectation is that speech in the EP is simpler, more neutral, and more utilitarian. The second component involves the identification of languages spoken in EP committee meetings using computational methods, to determine the language choices members of the EP make.

Feb 23 Andreas Obersteiner

Does 1/4 look larger than 1/3? The natural number bias in comparing symbolic and nonsymbolic fractions

When people compare the numerical values of fractions, they are often more accurate and faster when the larger fraction has the larger natural number components (e.g., 2/5 > 1/5) than when it has the smaller components (e.g., 1/3 > 1/4). However, recent studies produced conflicting evidence of this “natural number bias” when the comparison problems were more complex (e.g., 25/36 vs. 19/24). Moreover, it is unclear whether the bias also occurs when fractions are presented visually as shaded parts of rectangles rather than as numerical symbols. I will first present data from a reaction time study in which university students compared symbolic fractions. The results suggest that the occurrence and strength of the bias depends on the specific type of comparison problems and on people’s ability to activate overall fraction magnitudes. I will then present preliminary data from an eye tracking study in which university students compared rectangular fraction visualizations. Participants’ eye movements suggest that the pure presence of countable parts encouraged them to use unnecessary counting strategies, although the number of countable parts did not bias their decisions. The results have implications for mathematics education, which I will discuss in the talk.

Mar 2 Nicole Beckage, University of Kansas

Title: Multiplex network optimization to capture attention to features

Abstract: How does attention to features and current context affect people’s search in mental and physical spaces? I will examine how procedures for optimally searching through “multiplex” networks — networks with multiple layers or types of relationships — capture human search and retrieval patterns. Prior work on semantic memory, people’s memory for facts and concepts, has primarily focused on modeling similarity judgments of pairs of words as distances between points in a high-dimensional space (e.g., LSA by Laudauer et al, 1998; Word2Vec by Mikolov et al. 2013). While these decisions seem to accurately account for human similarity in some contexts, it’s very difficult to interpret high dimensional spaces, making it hard to use such representations for scientific research. Further, it is difficult to adapt these spaces to a specific context or task. Instead, I define a series of individual feature networks to construct a multiplex network, where each network in the multiplex captures a “sense” or type of similarity between items. I then optimize the “influence” of each of these feature networks within the multiplex framework, using real world search behavior on a variety of tasks. These tasks include semantic memory search in a cognitive task and information search in Wikipeida. The resulting weighting of the multiplex can capture aspects of human attention and contextual information in these diverse tasks. I explore how this method can provide interpretability to multi-relational data in psychology and other domains by developing an optimization framework that considers not only the presence or absence of relationships but also the nature of the relationships. While I focus on applications of semantic memory, I discuss mathematical proofs and simulation experiments that apply more generally to optimization problems in the multiplex network literature.

Mar 9 Psychology visit day Data Blitz! (WID Orchard Room)

The UW-Madison Psychology department will be hosting its second annual data blitz for prospective graduate students in the Orchard Room of the WID. The data blitz will feature speakers from across the department presenting their research in an easily digestible format. Each talk will be 5 minutes long with an additional 2 minutes for questions at the end of each talk. All are welcome to attend. Below you will find a list of the speakers and the titles of their talks:

    • Rista Plate “Unsupervised learning shifts emotion category boundaries”
    • Ron Pomper “Familiar object salience affects novel word learning”
    • Elise Hopman “Measuring meaning alignment between different languages”
    • Anna Bartel “How diagrams influence students’ mental models of mathematical story problems”
    • Aaron Cochrane “Chronic and phasic interactions between video game playing and addiction”
    • Pierce Edmiston “Correlations between programming languages and beliefs about programming”
    • Chris Racey “Neural processing underlying color preference judgments”
    • Sofiya Hupalo “Corticotropin-releasing factor (CRF) modulation of frontostriatal circuit function”

Mar 16 Varun Jog, Assistant Professor, ECE

Title: Mathematical models for social learning

Abstract: Individuals in a society learn about the world and form opinions not only through their own experiences, but also through interactions with other members in the society. This is an incredibly complex process, and although it is difficult to describe it completely using simple mathematical models, valuable insights may be obtained through such a study. Such social learning models have turned out to be a rich source of problems for probabilists, statisticians, information theorists, and economists. In this talk, we survey different social learning models, describe the necessary mathematical tools to analyze such models, and give examples of results that one may prove through such an approach.

Mar 23 Josh Cisler, Assistant Professor, Department of Psychiatry

Title: Real-time fMRI neurofeedback using whole-brain classifiers with an adaptive implicit emotion regulation task: analytic considerations

Most fMRI neuroimaging studies manipulate a psychological or cognitive variable (e.g., happy versus neutral faces) and observe the manipulations impact on brain function (e.g., amygdala activity is greater for happy faces). As such, the causal inferences that can be drawn from these studies is the effect of cognition on brain function, and not the effect of brain function on cognition. Real-time fMRI refers to processing of fMRI data simultaneous with data acquisition, enabling feedback of current brain states to be presented back to the participant in (near) real-time, thus enabling the participant to use the feedback signals to modify brain states. We are conducting an experiment using real-time fMRI neurofeedback where the feedback signal consists of classifier output (hyperplane distances) from a SVM trained on all grey matter voxels in the brain. The feedback signal is embedded within a commonly used implicit emotion regulation task, such that the task becomes easier or harder depending on the participant’s brain state. This type of ‘closed loop’ design allows for testing whether manipulations of brain state (via feedback) have a measurable impact on cognitive function (task performance). The purpose of this presentation will be to present the experimental design and resulting data properties for the purpose of obtaining feedback and recommendations for understanding and analyzing the complex dynamical systems relations between the feedback signal, brain state, and task performance.

Mar 30 (no meeting, spring break)

Apr 6 Student Research Presentations

Mini talk 1: Ayon Sen, Computer Sciences

For Teaching Perceptual Fluency, Machines Beat Human Experts

In STEM domains, students are expected to acquire domain knowledge from visual representations. Such learning requires perceptual fluency: the ability to intuitively and rapidly see what concepts visuals show and to translate among multiple visuals. Instructional problems that enhance perceptual fluency are highly influenced by sequence effects. Thus far, we lack a principled approach for identifying a sequence of perceptual-fluency problems that promote robust learning. Here, we describe a novel educational data mining approach that uses machine learning to generate an optimal sequence of visuals for perceptual-fluency problems. In a human experiment realted to chemistry, we show that a machine-generated sequence outperforms both a random sequence and a sequence generated by a human domain expert. To our knowledge, our study is the first to show that an educational data mining approach can yield desirable difficulties for perceptual learning.

Mini talk 2: Evan Hernandez, Ara Vartanian, Computer Sciences

Block-based programming environments are popular in computer science education, but the click-and-drag style of these environments render them inaccessible by students with motor impairments. Vocal user interfaces (VUIs) offer a popular alternative to traditional keyboard and mouse interfaces. We design a VUI for Google Blockly in the traditional Turtle/LOGOS setting and discuss the relevant design choices. We then investigate augmentations to educational programming environments. In particular, we describe a method of program synthesis for completing the partial or incorrect programs of students, and ask how educational software may leverage program synthesis to enhance student learning.

Apr 13 (no meeting)

Apr 20

Rob Nowak: All of Machine Learning

Apr 27 (STARTING AT 4PM instead of 3:45pm). Tyler Krucas, The Wisconsin Gaming Alliance.

An Industry Perspective on Data in Game Design and Development

The Wisconsin game development industry offers a surprisingly comprehensive cross section of the types of individuals and teams that develop video games. This includes everything from studios that collaborate on AAA titles such as Call of Duty and Bioshock Infinite, to studios that work largely on mobile or free-to-play games, to studios that primarily work on educational games or games for impact. In all cases, data collection and analysis is an important tool in every step of the game development process. However, the scale of the data collected and its use can vary dramatically from developer to developer. In my talk, I will provide an overview of the the game development ecosystem in Wisconsin, as well as examples of the different types data collection and use practices found in the regional industry. Critically, I’ll frame this discussion in the context of possible links with the HAMLET group – in terms of possible sources of data to address fundamental questions surrounding human learning or behavior as well as possible collaborations.

Fall 2017 Schedule:

Sept 15, Virtual and Physical: Two Frames of Mind, Bilge Mutlu (CS)

In creating interactive technologies, virtual and physical embodiments are often seen as two sides of the same coin. They utilize similar core technologies for perception, planning, and interaction and engage people in similar ways. Thus, designers consider these embodiments to be broadly interchangeable and choice of embodiment to primarily depend on the practical demands of an application. In this talk, I will make the case that virtual and physical embodiments elicit fundamentally different frames of mind in the users of the technology and follow different metaphors for interaction. These differences elicit different expectations, different forms of engagement, and eventually different interaction outcomes. I will discuss the design implications of these differences, arguing for different domains of interaction serving as appropriate context for virtual and physical embodiments.

October 13, Learning semantic representations for text: analysis of recent word embedding methods, Yingyu Liang (CS)

Recent advances in natural language processing build upon the approach of embedding words as low dimensional vectors. The fundamental observation that empirically justifies this approach is that these vectors can capture semantic relations. A probabilistic model for generating text is proposed to mathematically explain this observation and existing popular embedding algorithms. It also reveals surprising connections to classical notions such as Pointwise Mutual Information in computational linguistics, and allows to design novel, simple, and practical algorithms for applications such as embedding sentences as vectors.

October 20, Vlogging about research, Martina Rau (Ed Psych)

October 27, LUCID faculty meeting, No large group meeting

November 3, A Discussion of Open Science Practices, Martha W. Alibali (Psych)

This HAMLET session will be a discussion of open-science practices, led by Martha Alibali. We will start with brief discussion of the “replication crisis” and “questionable research practices”. We will then discuss solutions, including better research practices, data sharing and preregistration. Please read at least some of the provided papers, and come prepared to ask questions and share your experiences.

Replication crisis paper

Questionable Research Practices (QRPs) and solutions paper paper

Data sharing paper http://deevybee.blogspot.co.uk/2014/05/data-sharing-exciting-but-scary.html

Preregistration paper pic https://www.psychologicalscience.org/observer/seven-selfish-reasons-for-preregistration

November 10, Systematic misperceptions of 3D motion explained by Bayesian inference, Bas Rokers (Psych)

Abstract: Over the years, a number of surprising, but seemingly unrelated errors in 3D motion perception have been reported. Given the relevance of accurate motion perception to our everyday life, it is important to understand the cause of these perceptual errors. We considered that these perceptual errors might arise as a natural consequence of estimating motion direction given sensory noise and the geometry of 3D viewing. We characterized the retinal motion signals produced by objects moving along arbitrary trajectories through three dimensions and developed a Bayesian model of perceptual inference. The model predicted a number of known errors, including a lateral bias in the perception of motion trajectories, and a dependency of this bias on stimulus contrast and viewing distance. The model also predicted a number of previously unknown errors, including a dependency of perceptual bias on eccentricity, and a surprising tendency to misreport approaching motion as receding and vice versa. We then used standard 3D displays as well as a virtual reality (VR) headsets to test these predictions in naturalistic settings, and established that people make the predicted errors. In sum, we developed a quantitative model of 3D motion perception and provided a parsimonious account for a range of systematic perceptual errors in naturalistic environments.

November 17, Total variation regression under highly correlated designs, Becca Willett (ECE)

Abstract: I will describe a general method for solving high-dimensional linear inverse problems with highly correlated variables. This problem arises regularly in applications like neural decoding from fMRI data, where we often have two orders of magnitude more brain voxels than independent scans. Our approach leverages a graph structure that represents connections among voxels in the brain. This graph can be estimated from side sources, such as diffusion-weighted MRI, or from fMRI data itself. We will explore the underlying models, computational methods, and initial empirical results. This is joint work with Yuan Li and Garvesh Raskutti.

November 24 Thanksgiving Holiday

December 1, Micro-(Shape-And-Motion)-Scopes, Mohit Gupta (CS)

Imagine a drone looking for a safe landing site in a dense forest, or a social robot trying to determine the emotional state of a person by measuring her micro-saccade movements and skin-tremors due to pulse beats, or a surgical robot performing micro-surgery inside the body. In these applications, it is critical to resolve fine geometric details, such as tree twigs; to recover micro-motion due to biometric signals; and the precise motion of a robotic arm. Such precision is more than an order-of-magnitude beyond the capabilities of traditional vision techniques. I will talk about our recent work on designing extreme (micro) resolution 3D shape and motion sensors using unconventional, but low-cost optics, and computational techniques. These methods can measure highly subtle motions (< 10 microns), and highly detailed 3D geometry (<100 microns). These sensors can potentially detect a persons pulse or micro-saccade movements, and resolve fine geometric details such as a facial features, from a long distance.

December 8, Influence maximization in stochastic and adversarial settings, Po-Ling Loh (ECE)

We consider the problem of influence maximization in fixed networks, for both stochastic and adversarial contagion models. Such models may be used to model infection spreads in epidemiology, as well as the diffusion of information in viral marketing. In the stochastic setting, nodes are infected in waves according to linear threshold or independent cascade models. We establish upper and lower bounds for the influence of a subset of nodes in the network, where the influence is defined as the expected number of infected nodes at the conclusion of the epidemic. We quantify the gap between our upper and lower bounds in the case of the linear threshold model and illustrate the gains of our upper bounds for independent cascade models in relation to existing results. In the adversarial setting, an adversary is allowed to specify the edges through which contagion may spread, and the player chooses sets of nodes to infect in successive rounds. Our main result is to establish upper and lower bounds on the regret for possibly stochastic strategies of the adversary and player. This is joint work with Justin Khim (UPenn) and Varun Jog (UW-Madison).

For other events check out our calendar: Seminar and Events  This content and updates can be found: HAMLET

HAMLET Archives

Fall 2015 archive
Fall 2012 archive
Fall 2011 archive
Spring 2011 archive
Fall 2010 archive
Fall 2009 archive
Spring 2009 archive
Fall 2008 archive