Doğa Doğan, PhD

M. Doğa Doğan, PhDDoğa /ˈdoʊ.a/. The ğ is silent: say it like “Noah”, but with a D. Same for Doğan.

Designing a world AI can understand.

I am a Research Scientist at Adobe Research in Basel, Switzerland, working in the AI Experiences Lab. I received my PhD from Massachusetts Institute of Technology, where I worked in the Computer Science & Artificial Intelligence Laboratory. I build systems that make the world legible to AI: agent-driven tools for an AI-first information ecosystem, and unobtrusive metadata that gives physical objects a machine-readable identity.

Email: doga [at] {adobe.com, mit.edu, csail.mit.edu, acm.org, ieee.org}

Portrait of Mustafa Doğa Doğan

click a theme to filter publications

GenAI- and agent-driven tools that help people discover, evaluate, and act in an AI-first world.

A reinforcing loop between digital and physical contexts digital physical better understanding better experiences, anywhere
digital better understanding better experiences, anywhere physical

Unobtrusive metadata that gives everyday objects a reliable, machine-readable identity, beyond computer vision.

  • Natural markers

    Read intrinsic material and surface signatures.

    SensiCut (UIST '21), SpecTrack (SIGGRAPH Asia Posters '24)

    Best Poster Honorable Mention

  • Structural markers

    Hide data in fabrication artifacts and joints.

    G-ID (CHI '20), StructCode (SCF '23)

  • Interior markers

    Embed invisible, machine-only readable tags.

    InfraredTags (CHI '22), BrightMarker (UIST '23), InvisIto (UIST '26)

    Best Demo Honorable Mention

Publications

2026

Adobe Summit Sneaks '26 Adobe Summit 2026 Best Sneak Award

Project Face Off

Simulating A/B Testing Through Generative Audience Modeling and Predictive Evaluation

Abstract

A/B testing has become a fundamental method for optimizing digital experiences, yet conventional experimentation requires substantial traffic, time, and operational overhead before actionable insights emerge. We present Project Face Off, a generative AI system that simulates experimental outcomes by modeling prospective audience behavior and predicting comparative performance across design alternatives. Rather than relying exclusively on live deployment, the system synthesizes representative user personas from target audience characteristics and uses these simulated evaluators to assess competing content, interface, or campaign variants. Project Face Off estimates likely winning conditions, identifies factors contributing to performance differences, and generates natural-language explanations that communicate predicted outcomes and decision rationales. This approach shifts experimentation from a purely retrospective process toward a proactive design tool, and helps reduce the cost of iteration while supporting more informed creative and strategic decisions.

Featured in Adobe Research News, The AI Economy and B&T Australia

UIST '26

UXCascade

Scalable Usability Testing with Simulated User Agents

Abstract

Simulated user agents are increasingly deployed in usability testing to support fast, iterative UX workflows, as they generate rich data such as action logs and think-aloud reasoning, but the unstructured nature of this output often obscures actionable insights. We present UXCascade, an interactive tool for extracting, aggregating, and presenting agent-generated usability feedback at scale. Our core contribution is a multi-level analysis workflow that (1) highlights patterns across persona traits, goals, and outcomes, (2) links agent reasoning to specific issues, and (3) supports actionable design improvements. UXCascade operationalizes this approach by listing agent goals, traits, and issues in a structured overview. Practitioners can explore detailed reasoning traces and annotated views, propose interface edits, and assess their impact across personas. This enables a top-down, exploration-driven analysis from patterns to concrete, actionable UX interventions. A user study with eight UX professionals demonstrates that UXCascade integrates into existing workflows, enabling iterative feedback during early-stage interface development.

* The authors contributed equally and jointly supervised this work

UIST '26

InvisIto

Weaving Unobtrusive Infrared Markers for Ubiquitous Textile Interaction

Abstract

Textiles are increasingly explored as media for interacting with digital information. However, many of the existing approaches rely on visible tags, printed overlays, or electronic modules that compromise the fabric’s aesthetic and tactile qualities. To address this, we present InvisIto, a method for weaving visually unobtrusive yet machine-readable infrared markers directly into fabrics using near-infrared (NIR)-absorbing yarns. Although these yarns look similar to standard fibers in ambient light, they produce strong contrast in NIR imaging. Our method includes: (1) a design tool that helps users easily embed infrared markers into weaving drafts, (2) five disguising strategies that further reduce marker visibility under ambient light, and (3) a camera-based detection pipeline for decoding and tracking the woven markers. InvisIto supports both woven QR codes for data encoding and woven ArUco markers for binary input and deformation tracking.

We demonstrate applications across hand weaving, Jacquard weaving, and industrial fabrication, showing that InvisIto supports scalable interaction and fabrication from bespoke artifacts to mass production.

IUI '26

UI Remix

Supporting UI Design Through Interactive Example Retrieval and Remixing

Abstract

Designing user interfaces (UIs) is a critical step when launching products, building portfolios, or personalizing projects, yet end users without design expertise often struggle to articulate their intent and to trust design choices. Existing example-based tools either promote broad exploration, which can cause overwhelm and design drift, or require adapting a single example, risking design fixation. We present UI Remix, an interactive system that supports mobile UI design through an example-driven design workflow. Powered by a multimodal retrieval-augmented generation (MMRAG) model, UI Remix enables iterative search, selection, and adaptation of examples at both the global (whole interface) and local (component) level. To foster trust, it presents source transparency cues such as ratings, download counts, and developer information. In an empirical study with 24 end users, UI Remix significantly improved participants’ ability to achieve their design goals, facilitated effective iteration, and encouraged exploration of alternative designs. Participants also reported that source transparency cues enhanced their confidence in adapting examples. Our findings suggest new directions for AI-assisted, example-driven systems that empower end users to design with greater control, trust, and openness to exploration.

Teaser figure for XAgen

Design & AI '26 · CHI '26 Poster

XAgen

Designing Explainability Support for Failure Detection and Correction in Multi-Agent Workflows

Abstract

The rapid adoption of LLM-powered multi-agent systems has brought users with diverse AI expertise—from software developers to domain experts—into the process of building and maintaining complex agentic workflows. Yet when these distributed systems fail, their non-deterministic and opaque nature makes it difficult for users to observe, attribute, and correct errors. To understand practitioners’ needs, we conducted formative interviews with 12 practitioners spanning AI developers who build agentic systems to domain experts who utilize them, surfacing five key challenge themes and four design goals for explainability tools. Building on these insights, we designed XAgen, an explainability interface that supports failure detection and correction through three capabilities: log visualization for glanceable workflow understanding, human-in-the-loop feedback to capture expert judgment, and automatic error identification via an LLM-as-a-judge. In a user evaluation with 8 participants, XAgen was rated significantly higher than a state-of-the-art observability baseline across all four dimensions—understanding, usability, helpfulness, and satisfaction—with median scores of 6–7 out of 7 (all p < 0.05). All three features were rated useful: log visualization received the highest helpfulness rating (median = 5/5, p < 0.01), followed by automatic error identification (median = 4.5/5, p < 0.05) and human-in-the-loop output feedback (median = 4/5, p = 0.07). Our findings yield empirically grounded design guidelines for explainable agentic AI and highlight open challenges for context-aware interactive debugging of multi-agent workflows.

2025

SCF '25

TinkerXR

In-Situ, Reality-Aware CAD and 3D Printing Interface for Novices

Abstract

Despite the growing accessibility of augmented reality (AR) for visualization, existing computer-aided design (CAD) systems remain confined to traditional screens or require complex setups or predefined parameters, limiting immersion and accessibility for novices. We present TinkerXR, an open source interface enabling in-situ design and fabrication through Constructive Solid Geometry (CSG) modeling. TinkerXR operates solely with a headset and 3D printer, allowing users to design directly in and for their physical environments. By leveraging spatial awareness, depth occlusion, recognition of physical constraints, reference objects, and hand movement controls, TinkerXR enhances realism, precision, and ease of use. Its AR-based workflow integrates design and 3D printing with a drag-and-drop interface for a 3D printer’s virtual twin. A user study comparing TinkerXR with Tinkercad demonstrates higher accessibility, engagement, and ease of use for novices. By bridging the gap between digital creation and physical output, TinkerXR aims to transform everyday spaces into accessible and expressive creative studios.

Teaser figure for Programmable Reality

Frontiers in VR '25

Programmable Reality

Abstract

Innovations in spatial computing and artificial intelligence (AI) are making it possible to overlay dynamic, interactive digital elements on the physical world. Soon, every object might have a real-time digital twin, enabling the “Internet of Things” so as to identify and interact with even unconnected items. This programmable reality would enable computational manipulation of the world around us through alteration of its appearance or functionality, similar to software, but for reality itself. Augmented reality (AR) provides the ideal bridge between AI output and human input in the physical world. As AR devices become ubiquitous and indistinguishable from reality, these technologies blur the lines between reality and simulations. This raises profound questions about how we perceive and experience the world while having implications for memory, learning, and even behavior. Programmable reality enabled by AR and AI has vast potential to reshape our relationships with the digital realm, ultimately making it an extension of the physical realm.

RA-L '25

RAMPA

Robotic Augmented Reality for Machine Programming by Demonstration

Abstract

This paper introduces Robotic Augmented Reality for Machine Programming by Demonstration (RAMPA), the first ML-integrated, XR-driven end-to-end robotic system, allowing training and deployment of ML models such as ProMPs on the fly, and utilizing the capabilities of state-of-the-art and commercially available AR headsets, e.g., Meta Quest 3, to facilitate the application of Programming by Demonstration (PbD) approaches on industrial robotic arms, e.g., Universal Robots UR10. Our approach enables in-situ data recording, visualization, and fine-tuning of skill demonstrations directly within the user’s physical environment. RAMPA addresses critical challenges of PbD, such as safety concerns, programming barriers, and the inefficiency of collecting demonstrations on the actual hardware. The performance of our system is evaluated against the traditional method of kinesthetic control in teaching three different robotic manipulation tasks and analyzed with quantitative metrics, measuring task performance and completion time, trajectory smoothness, system usability, user experience, and task load using standardized surveys. Our findings indicate a substantial advancement in how robotic tasks are taught and refined, promising improvements in operational safety, efficiency, and user engagement in robotic programming.

CHI '25

Imprinto

Enhancing Infrared Inkjet Watermarking for Human and Machine Perception

Abstract

Hybrid paper interfaces leverage augmented reality to combine the desired tangibility of paper documents with the affordances of interactive digital media. Typically, virtual content can be embedded through direct links (e.g., QR codes); however, this impacts the aesthetics of the paper print and limits the available visual content space. To address this problem, we present Imprinto, an infrared inkjet watermarking technique that allows for invisible content embeddings only by using off-the-shelf IR inks and a camera. Imprinto was established through a psychophysical experiment, studying how much IR ink can be used while remaining invisible to users regardless of background color. We demonstrate that we can detect invisible IR content through our machine-learning pipeline, and we developed an authoring tool that optimizes the amount of IR ink on the color regions of an input document for machine and human detectability. Finally, we demonstrate several applications, including augmenting paper documents and objects.

CHI '25

XR-penter

Material-Aware and In Situ Design of Scrap Wood Assemblies

Abstract

Woodworkers have to navigate multiple considerations when planning a project, including available resources, skill-level, and intended effort. Do it yourself (DIY) woodworkers face these challenges most acutely because of tight material constraints and a desire for custom designs tailored to specific spaces. To address these needs, we present XR-penter, an **extended reality (XR) application that supports in situ, material-aware woodworking for casual makers. Our system enables users to design virtual scrap wood assemblies directly in their workspace, encouraging sustainable practices through the use of discarded materials. Users register physical material as virtual twins, manipulate these twins into an assembly in XR, and preview cuts needed for fabrication. We conducted a case study and feedback sessions to demonstrate how XR-penter supports improvisational workflows in practice, the type of woodworker who would benefit most from our system, and insights on integrating similar spatial and material considerations into future work.

CHI '25

Draw2Cut

Direct On-Material Annotations for CNC Milling

Abstract

Creating custom artifacts with computer numerical control (CNC) milling machines typically requires mastery of complex computer-aided design (CAD) software. To eliminate this user barrier, we introduced Draw2Cut, a novel system that allows users to design and fabricate artifacts by sketching directly on physical materials. Draw2Cut employs a custom-drawing language to convert user-drawn lines, symbols, and colors into toolpaths, thereby enabling users to express their creative intent intuitively. The key features include real-time alignment between material and virtual toolpaths, a preview interface for validation, and an open-source platform for customization. Through technical evaluations and user studies, we demonstrate that Draw2Cut lowers the entry barrier for personal fabrication, enabling novices to create customized artifacts with precision and ease. Our findings highlight the potential of the system to enhance creativity, engagement, and accessibility in CNC-based woodworking.

CHI '25

FontCraft

Multimodal Font Design Using Interactive Bayesian Optimization

Abstract

Creating new fonts requires a lot of human effort and professional typographic knowledge. Despite the rapid advancements of automatic font generation models, existing methods require users to prepare pre-designed characters with target styles using font-editing software, which poses a problem for non-expert users. To address this limitation, we propose FontCraft, a system that enables font generation without relying on pre-designed characters. Our approach integrates the exploration of a font-style latent space with human-in-the-loop preferential Bayesian optimization and multimodal references, facilitating efficient exploration and enhancing user control. Moreover, FontCraft allows users to revisit previous designs, retracting their earlier choices in the preferential Bayesian optimization process. Once users finish editing the style of a selected character, they can propagate it to the remaining characters and further refine them as needed. The system then generates a complete outline font in OpenType format. We evaluated the effectiveness of FontCraft through a user study comparing it to a baseline interface. Results from both quantitative and qualitative evaluations demonstrate that FontCraft enables non-expert users to design fonts efficiently.

DIS '25 WiP

ImprovMate

Multimodal AI Assistant for Improv Actor Training

Abstract

Improvisation training for actors presents unique challenges, particularly in maintaining narrative coherence and managing cognitive load during performances. Previous research on AI in improvisation performance often predates advances in large language models (LLMs) and relies on human intervention. We introduce ImprovMate, which leverages LLMs as GPTs to automate the generation of narrative stimuli and cues, allowing actors to focus on creativity without keeping track of plot or character continuity. Based on insights from professional improvisers, ImprovMate incorporates exercises that mimic live training, such as abrupt story resolution and reactive thinking exercises, while maintaining coherence via reference tables. By balancing randomness and structured guidance, ImprovMate provides a groundbreaking tool for improv training. Our pilot study revealed that actors might embrace AI techniques if the latter mirrors traditional practices, and appreciate the fresh twist introduced by our approach with the AI-generated cues.

2024

UIST '24 2025 SXSW Innovation Awards Finalist

XR-Objects

Augmented Object Intelligence with XR-Objects

Abstract

Seamless integration of physical objects as interactive digital entities remains a challenge for spatial computing. This paper explores Augmented Object Intelligence (AOI) in the context of XR, an interaction paradigm that aims to blur the lines between digital and physical by equipping real-world objects with the ability to interact as if they were digital, where every object has the potential to serve as a portal to digital functionalities. Our approach utilizes real-time object segmentation and classification, combined with the power of Multimodal Large Language Models (MLLMs), to facilitate these interactions without the need for object pre-registration. We implement the AOI concept in the form of XR-Objects, an open-source prototype system that provides a platform for users to engage with their physical environment in contextually relevant ways using object-based context menus. This system enables analog objects to not only convey information but also to initiate digital actions, such as querying for details or executing tasks. Our contributions are threefold: (1) we define the AOI concept and detail its advantages over traditional AI assistants, (2) detail the XR-Objects system’s open-source design and implementation, and (3) show its versatility through various use cases and a user study.

Featured in Google Research Blog

UIST '24 Best Demo Honorable Mention

Speed-Modulated Ironing

High-Resolution Shade and Texture Gradients in Single-Material 3D Printing

Abstract

We present Speed-Modulated Ironing, a new fabrication method for programming visual and tactile properties in single-material 3D printing. We use one nozzle to 3D print and a second nozzle to reheat printed areas at varying speeds, controlling the material’s temperature-response. The rapid adjustments of speed allow for fine-grained reheating, enabling high-resolution color and texture variations. We implemented our method in a tool that allows users to assign desired properties to 3D models and creates corresponding 3D printing instructions. We demonstrate our method with three temperature-responsive materials: a foaming filament, a filament with wood fibers, and a filament with cork particles. These filaments respond to temperature by changing color, roughness, transparency, and gloss. Our technical evaluation reveals the capabilities of our method in achieving sufficient resolution and color shade range that allows surface details such as small text, photos, and QR codes on 3D-printed objects. Finally, we provide application examples demonstrating the new design capabilities enabled by Speed-Modulated Ironing.

Featured in MIT News, TU Delft News, 3D Printing Industry and 3DPrint.com

CHI '24

MoiréWidgets

High-Precision, Passive Tangible Interfaces via Moiré Effect

Abstract

We introduce MoiréWidgets, a novel approach for tangible interaction that harnesses the Moiré effect—a prevalent optical phenomenon—to enable high-precision event detection on physical widgets. Unlike other electronics-free tangible user interfaces which require close coupling with external hardware, MoiréWidgets can be used at greater distances while maintaining high-resolution sensing of interactions. We define a set of interaction primitives, e.g., buttons, sliders, and dials, which can be used as standalone objects or combined to build complex physical controls. These consist of 3D printed structural mechanisms with patterns printed on two layers—one on paper and the other on a plastic transparency sheet—which create a visual signal that amplifies subtle movements, enabling the detection of user inputs. Our technical evaluation shows that our method outperforms standard fiducial markers and maintains sub-millimeter accuracy at 100 cm distance and wide viewing angles. We demonstrate our approach by creating an audio console and indicate how our approach could extend to other domains.

SIGGRAPH Asia '24 Poster Best Poster Honorable Mention

SpecTrack

Learned Multi-Rotation Tracking via Speckle Imaging

Abstract

Precision pose detection is increasingly demanded in fields such as personal fabrication, Virtual Reality (VR), and robotics due to its critical role in ensuring accurate positioning information. However, conventional vision-based systems used in these systems often struggle with achieving high precision and accuracy, particularly when dealing with complex environments or fast-moving objects. To address these limitations, we investigate Laser Speckle Imaging (LSI), an emerging optical tracking method that offers promising potential for improving pose estimation accuracy. Specifically, our proposed LSI-Based Tracking leverages the captures from a lensless camera and a retro-reflector marker with a coded aperture to achieve multi-axis rotational pose estimation with high precision. Our extensive trials using our in-house built testbed have shown that SpecTrack achieves an accuracy of 0.31° (std=0.43°) , significantly outperforming state-of-the-art approaches and improving accuracy up to 200%.

2023

UIST '23

BrightMarker

3D Printed Fluorescent Markers for Object Tracking

Abstract

Existing invisible object tagging methods are prone to low resolution, which impedes tracking performance. We present BrightMarker, a fabrication method that uses fluorescent filaments to embed easily trackable markers in 3D printed color objects. By using an infrared-fluorescent filament that “shifts” the wavelength of the incident light, our optical detection setup filters out all the noise to only have the markers present in the infrared camera image. The high contrast of the markers allows us to track them robustly regardless of the moving objects’ surface color.

We built a software interface for automatically embedding these markers for the input object geometry, and hardware modules that can be attached to existing mobile devices and AR/VR headsets. Our image processing pipeline robustly localizes the markers in real-time from the captured images. BrightMarker can be used in a variety of applications, such as custom fabricated wearables for motion capture, tangible interfaces for AR/VR, rapid product tracking, and privacy-preserving night vision. BrightMarker exceeds the detection rate of state-of-the-art invisible marking, and even small markers (1″x1″) can be tracked at distances exceeding 2m.

Featured in MIT News, Hackster.io, Hackaday and SwissCognitive

SCF '23

StructCode

Leveraging Fabrication Artifacts to Store Data in Laser-Cut Objects

Abstract

We introduce StructCode, a technique to store machine-readable data in laser-cut objects using their fabrication artifacts. StructCode modifies the lengths of laser-cut finger joints and/or living hinges to represent bits of information without introducing additional parts or materials. We demonstrate StructCode through use cases for augmenting laser-cut objects with data such as labels, instructions, and narration. We present and evaluate a tag decoding pipeline that is robust to various backgrounds, viewing angles, and wood types. In our mechanical evaluation, we show that StructCodes preserve the structural integrity of laser-cut objects.

Featured in MIT News and Hackster.io

CHI '23 LBW

StandARone

Infrared-Watermarked Documents as Portable Containers of AR Interaction and Personalization

Abstract

Hybrid paper interfaces leverage augmented reality (AR) to combine the desired tangibility of paper documents with the affordances of interactive digital media. Typically, the instructions for how the virtual content should be generated are not an intrinsic part of the document but rather accessed through a link to remote resources. To enable hybrid documents to be portable containers of also the AR content, we introduce StandARone documents. Using our system, a document author can define AR content and embed it invisibly on the document using a standard inkjet printer and infrared-absorbing ink. A document consumer can interact with the embedded content using a smartphone with a NIR camera without requiring a network connection. We demonstrate several use cases of StandARone including personalized offline menus, interactive visualizations, and location-aware packaging.

2022

CHI '22 Best Demo Honorable Mention

InfraredTags

Invisible AR Markers & Barcodes Using Low-Cost, Infrared-Based 3D Printing & Imaging Tools

Abstract

Existing approaches for embedding unobtrusive tags inside 3D objects require either complex fabrication or high-cost imaging equipment. We present InfraredTags, which are 2D codes and markers imperceptible to the naked eye that can be 3D printed as part of objects, and detected rapidly by low-cost near-infrared cameras. InfraredTags achieve this by being printed from an infrared-transmitting filament which infrared cameras can see through, and by having air gaps inside for the tag’s bits which infrared cameras capture as darker pixels in the image. We built a user interface that facilitates the integration of common tags (QR codes, ArUco markers) with the object geometry to make them 3D printable as InfraredTags. We also developed a low-cost infrared imaging module that augments existing mobile devices and decodes tags using our image processing pipeline. We demonstrate how our method enables applications, such as object tracking and embedding metadata for augmented reality and tangible interactions.

Featured in Popular Science, New Scientist and MIT News

2021

UIST '21

SensiCut

Material-Aware Laser Cutting Using Speckle Sensing and Deep Learning

Abstract

Laser cutter users face difficulties distinguishing between visually similar materials. This can lead to problems, such as using the wrong power/speed settings or accidentally cutting hazardous materials. To support users in identifying the sheets, we present SensiCut, a material sensing platform for laser cutters. In contrast to approaches that detect the appearance of the material with a conventional camera, SensiCut identifies the material by its surface structure using speckle sensing and deep learning. SensiCut comes with a compact hardware add-on for the laser cutter and a user interface that integrates material sensing into the cutting workflow. In addition to improving the traditional workflow, SensiCut enables new applications, such as automatically partitioning the design when engraving on multi-material objects or adjusting the shape of the design based on the kerf of the identified material. We evaluate SensiCut’s accuracy for different types of materials under different conditions, such as with various illuminations and sheet orientations.

Featured in The Next Web, Photonics.com and MIT News

2020

CHI '20

G-ID

Identifying 3D Prints Using Slicing Parameters

Abstract

G-ID is a method that utilizes the subtle patterns left by the 3D printing process to distinguish and identify objects that otherwise look similar to the human eye. The key idea is to mark different instances of a 3D model by varying slicing parameters that do not change the model geometry but can be detected as machine-readable differences in the print. As a result, G-ID does not add anything to the object but exploits the patterns appearing as a byproduct of slicing, an essential step of the 3D printing pipeline. We introduce the G-ID slicing & labeling interface that varies the settings for each instance, and the G-ID mobile app, which uses image processing techniques to retrieve the parameters and their associated labels from a photo of the 3D printed object. Finally, we evaluate our method’s accuracy under different lighting conditions, when objects were printed with different filaments and printers, and with pictures taken from various positions and angles.

Featured in 3DPrint.com, Hackster.io and ITmedia

UIST '20 Best Demo Honorable Mention

DefeXtiles

3D Printing Quasi-Woven Fabric via Under-Extrusion

Abstract

We present DefeXtiles, a rapid and low-cost technique to produce tulle-like fabrics on unmodified fused deposition modeling (FDM) printers. The under-extrusion of filament is a common cause of print failure, resulting in objects with periodic gap defects. In this paper, we demonstrate that these defects can be finely controlled to quickly print thinner, more flexible textiles than previous approaches allow. Our approach allows hierarchical control from micrometer structure to decameter form and is compatible with all common 3D printing materials. In this paper, we introduce the mechanism of DefeXtiles and establish the design space through a set of primitives with detailed workflows. We demonstrate the interactive features and new use cases of our approach through a variety of applications, such as fashion design prototyping, interactive objects, aesthetic patterning, and single-print actuators.

Featured in Gizmodo and MIT News

2019

CHI '19

FoldTronics

Creating 3D Objects with Integrated Electronics Using Foldable Honeycomb Structures

Abstract

FoldTronics is a 2D-cutting based fabrication technique to integrate electronics into 3D folded objects. The key idea is to cut and perforate a 2D sheet to make it foldable into a honeycomb structure using a cutting plotter; before folding the sheet into a 3D structure, users place the electronic components and circuitry onto the sheet. The fabrication process only takes a few minutes enabling users to rapidly prototype functional interactive devices. The resulting objects are lightweight and rigid, thus allowing for weight-sensitive and force-sensitive applications. Finally, due to the nature of the honeycomb structure, the objects can be folded flat along one axis and thus can be efficiently transported in this compact form factor. We describe the structure of the foldable sheet, and present a design tool that enables users to quickly prototype the desired objects. We showcase a range of examples made with our design tool, including objects with integrated sensors and display elements.

Featured in Hackster.io

2017

ICRA '17 Best Medical Robotics Paper Award Nomination

Soft Capsule Endoscope

Magnetically Actuated Soft Capsule Endoscope for Fine-Needle Aspiration

Abstract

This paper presents a magnetically actuated soft capsule endoscope for fine-needle aspiration biopsy (B-MASCE) in the upper gastrointestinal tract. A thin and hollow needle is attached to the capsule, which can penetrate deeply into tissues to obtain subsurface biopsy sample. The design utilizes a soft elastomer body as a compliant mechanism to guide the needle. An internal permanent magnet provides a means for both actuation and tracking. The capsule is designed to roll towards its target and then deploy the biopsy needle in a precise location selected as the target area. B-MASCE is controlled by multiple custom-designed electromagnets while its position and orientation are tracked by a magnetic sensor array.

Featured in Engadget and IEEE Spectrum

About

My research is driven by a fundamental shift in how people access information: we are moving beyond websites and search engines toward LLMs, personal AI agents, and spatial computing platforms. In this new paradigm, AI systems become the primary interface to knowledge. How can these emerging interfaces deliver reliable, context-aware information anywhere, at any moment?

I investigate this question through the lens of grounded ubiquitous intelligence: designing systems, objects, and infrastructures that make the world legible to AI. My work spans two complementary directions.

1. GenAI- and agent-driven tools for next-generation information access. At Adobe, I design systems that help organizations and creators adapt to an AI-first information ecosystem. This includes Adobe LLM Optimizer and Sites Optimizer, which support content quality and discoverability in LLM-dominated environments, i.e., the emerging field of generative engine optimization (GEO). I also develop interfaces for agentic AI and synthetic personas, such as Adobe's Project Face Off (Adobe Summit '26 Best Sneak award) and Project Get Savvy, and explore multimodal, contextual AI+AR interactions, e.g., augmented object intelligence.

2. Metadata-driven intelligence and interaction for everyday, real-world objects. During my PhD at MIT CSAIL, I developed new ways for physical objects to carry unobtrusive metadata that AI systems can interpret with high reliability. This line of work, ubiquitous metadata, includes systems such as G-ID, InfraredTags, BrightMarker, and Imprinto, which embed machine-readable information directly into materials. This allows AI and AR systems to perceive objects with guarantees that vision-only algorithms cannot achieve.

Across both directions, my goal is to build AI systems that support the future of information access and fluidly connect digital and physical contexts, as part of a broader shift toward truly ubiquitous and grounded intelligence.

I regularly collaborate with interdisciplinary teams and present my findings at major computer science and human-computer interaction (HCI) venues. To discuss projects or collaborations, please reach out by email.

Experience

  1. Research Scientist, Adobe

    AI Experiences Lab · Research Division

    Aug 2024 - Present · Basel, Switzerland

  2. Visiting Postdoctoral Researcher, University of Tokyo

    User Interface Research Group · Graduate School of Information Science and Technology

    Collaborator: Takeo Igarashi

    Jun 2024 - Jul 2024 · Tokyo, Japan

  3. Postdoctoral Researcher & Lecturer, Boğaziçi University

    Cognitive Learning and Robotics (COLORS) Lab · Computer Engineering Department

    Collaborator: Emre Ugur

    Feb 2024 - Jul 2024 · Istanbul, Turkey

  4. PhD Student Researcher, Google

    Augmented Reality (AR/VR) · Research Division

    Advisors: David Kim, Mar Gonzalez-Franco

    Oct 2023 - Mar 2024 · Zurich, Switzerland

  5. Visiting Postdoctoral Researcher, TU Delft

    Department of Sustainable Design Engineering · Faculty of Industrial Design Engineering

    Collaborator: Zjenja Doubrovski

    May 2023 - Jul 2023 · Delft, Netherlands

  6. Research Scientist Intern, Adobe

    Document Intelligence Lab · Research Division

    Advisors: Alexa Siu, Tong Sun

    May 2022 - Nov 2022 · San Jose, CA

  7. Research Assistant, Massachusetts Institute of Technology (MIT)

    Computer Science and Artificial Intelligence Lab (CSAIL) · HCI Engineering Group

    Advisor: Stefanie Mueller

    Aug 2018 - Dec 2022 · Cambridge, MA

  8. Visiting Researcher, Massachusetts Institute of Technology (MIT)

    Computer Science and Artificial Intelligence Lab (CSAIL) · HCI Engineering Group

    Advisor: Stefanie Mueller

    Jun 2017 - Sep 2017 · Cambridge, MA

  9. Undergraduate Researcher, UCLA

    Electrical and Computer Engineering · Laboratory for Embedded Machines and Ubiquitous Robots

    Advisor: Ankur Mehta

    2017 · Los Angeles, CA

  10. Undergraduate Researcher & Teaching Assistant, Boğaziçi University

    Haptics & Robotics Lab · Intelligent Systems Lab

    Advisors: Evren Samur, Işıl Bozma

    • EE142: Introduction to Digital Systems (Fall 2016 & Fall 2017)

    Oct 2016 - Dec 2017 · Istanbul, Turkey

  11. Research Assistant, Max Planck Institute for Intelligent Systems

    Physical Intelligence Department · Medical Millirobots Group

    Advisor: Metin Sitti

    Jun 2016 - Aug 2016 · Stuttgart, Germany

Education

  1. Ph.D. in Computer Science

    Massachusetts Institute of Technology (MIT)

    Thesis: Ubiquitous Metadata: Design and Fabrication of Embedded Markers for Real-World Object Identification and Interaction

    2018-2024 · Cambridge, MA

  2. M.Sc. in Electrical Engineering & Computer Science

    Massachusetts Institute of Technology (MIT)

    Past chair of the MIT European Club and Career Fair (2020-21)

    2018-2020 · Cambridge, MA

  3. B.Sc. in Electrical & Electronics Engineering

    Boğaziçi University

    Past chair of the IEEE Student Branch (2015-16)

    2014-2018 · Istanbul, Turkey

  4. Exchange student, Electrical and Computer Engineering

    University of California, Los Angeles (UCLA)

    2017 · Los Angeles, CA

Invited talks

This list is not up to date.

  • MIT Sloan, Machine Intelligence for Manufacturing and Operations (MIMO) Research Forum (Dec ’22)
  • University of Maryland HCIL BBL Speaker Series (Oct ’22)
  • Samsung America Research, Mountain View (Jul ’22)
  • UC San Diego Design Lab (Aug ’22)
  • Stanford University HCI Lunch Seminar (Jun ’22)
  • University College London, Virtual Graphics and Computer Graphics Seminar Series (Feb ’22)
  • MIT Open Learning, MIT Horizon Talks (Mar ’22)
  • University of Chicago, Human-Computer Integration Lab (Sep ’19)

Service

  • Steering Committee Member: ACM SCF
  • Conference Chair: ACM SCF (’26)
  • Posters Chair: ACM UIST (’22, ’23)
  • Student Volunteer Chair: ACM UIST (’19, ’20)
  • Social Events Chair: ACM UIST (’21)
  • Organizer: ACM CHI Personal Fabrication Community Event (’20)
  • Subcommittee Chair, ACM DIS, “Artifacts and Systems” (’26)
  • Associate Chair, ACM CHI (’27; ’26, “Blending Interaction: Engineering Interactive Systems & Tools” subcommittee)
  • Associate Chair, ACM UIST (’25, ’26)
  • Associate Chair, ACM TEI (’24)
  • ACM CHI (’20, ’21, ’22, ’23)
  • ACM UIST (’21, ’22)
  • ACM SIGGRAPH Asia (’22)
  • IEEE ICRA (’21)
  • ACM TEI (’21)
  • IEEE ISMAR (’20)
  • ACM EICS (’21)
  • ACM IDC WIP (’21)
  • IEEE Sensors Journal (’20, ’21)

Moments