Holly Huey.

Applied Scientist @ Adobe | Experimental Psychology, PhD | Conducting large-scale evaluations of GenAI image & video systems

Hi there! I'm an Applied Scientist at Adobe in San Francisco. My research focuses on developing large-scale human behavioral benchmarks in order to evaluate and improve GenAI image & video systems.

I received my PhD in Experimental Psychology from the University of California San Diego, under Dr. Judith Fan @ The Cognitive Tools Lab.

Broadly, I study how people create, interpret, and interact with visual content. By conducting a wide range of custom coded web experiments, I crowdsource datasets of up to >90K human responses in order to systematically measuring people's visual content creation. This scale of data enables me to statistically evaluate GenAI algoritms, heuristic models, and LLMs against human behavior and communication.

My disseration research specialized in understanding user intentions in visual content creation—specifically, how content creators' different goals to make diverse visual content directly impacts:

  • what their output content will look like
  • how viewers will interpret that output depending on their own interpretation goals

  • I evaluated these questions by developing and analyzing large-scale drawing, diagram, and data visualization datasets.

    research domains

    VISUALIZATIONS

    My PhD research evaluates human visual content creation in the form of drawings, diagrams, and data visualizations to investigate how people interpret the objects and scenes in their environments.

    IMAGES

    Text-to-image generation models provide new opportunities for people to quickly & flexibly create images. To systematically evaluate model errors, I developed a series of behavioral tasks to crowdsource human annotators and generate quality scores for image datasets.

    VIDEO

    At Adobe Research, I've conducted qualitative interviews aimed at discovering user needs & goals and custom coded online experiments for quantitatively evaluating user behaviors related to video production and editing.
    I have also explored & published work in other domains: language, social pragmatics (theory of mind), navigation, causal perception, symbolic reasoning, and developmental psychology. See below publications or my Google Scholar for papers in these areas.

    Designed by Holly Huey. Photo credit: Petrina Chan (headshot) & Terra Huey (headshot art)